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Adele aoldberf, and Patrick Suppes 

Qwr earlier research in the desipin of instructional systems and 
currlculimis for teachlnp; mathematical logic to gifted elementary school 
stiide- ts has been extended to the teaching of university-le.vel courses 
(uoldVer/j;^ 1975; Suppes, 1972; 3uppes^ Ihrke, 1970). In this report, 
we describe the curriculum and problem types of a canputer- based course 
offered at Stanford University: Philosophy 'VIA, Introduction ^o Symbolic 
Loi^ic. We base our description on an analysis of the work of 79 students 
I'ata on these students were collected d\iring the third and fourth quar- 
ters (sprin,-; and fall, 1975) in which the course was offered. The in- 
str-;ctioual pror^ram wa& wi^itten in LISP 1.'; for the DEC PDP-10 at the 
Institute for Ma-chematical Studies in the Social Sciences (IMGSS). Pro- 
fit arriminf:, details of the computer-based system, proof checker, and lesson 
driver are provided elsevAe (Goldberg, 1975> 197^). 

Course Description 

The main objective of the Stanford logic course is to lairiliarize 
the student with an exact and complete theory of logical inference. 
The course is taught solely by computer; all material is presented on 
the terminal and all problems are solved through interactions with a 
mechanical proof checker. Seminars, with optional attendance, were held 
several times during the spring quarter to discuss special topics. At- 
tendance was low, so seminars were not held in the fall. 
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No book was reqi.ireti, althoi^-Vi c.ppr ; . caco+ert Introduction 

to Logic fey Guppes (19'yY) were recorn/neiiderU Depending on the class size, 
one or two teaciiin^: assistants, usually graduate students in the Philos- 
ophy Department, were available 9 ho\irs each week ro answer cuestions 
arlsinp; from the computer-based curriculum. A research programmer was 
also available* 

An outline of the course is shown in Table I* Problems given the 
students emphasi^.e proving arguments valid by constructing proofs in a 

Insert Table I about here 

natural deduction system (Lessons UOl-1+08), or provin-; arguments invalid 
by either the method of truth analysis (Lessons U05 and k09) or of inter* 
pro^^atior (Lessons ii23 and ^28), The method of interpretation is also 

a]:, liea to prove premises consistent, or axioms of a theory independent 

Heginning with Lesson kl3, examples of axlomatically formulated 
thr-or i«js are Introdiiced. Two examples, the elementary theory of Abelian 
;;rout.s and the elementary theory of non-Abelian groups, are given in 
Lessons '+15 through 420, The axioms and theorems studied in these les- 
sons are listed in Table II • Numerous other examples, in the form of 

Insert Table II about here 

flndlrg-axioras exercises, range from the algebra of real numbers to a 

segment of elementary geometry* 

Les^son U21 teaches the student how to do the finding-axioms exercises: 
how to specify a set of axioms from a given list of statements, and how to 
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problem? 



Uoi 19 .yi v;-vll- :\v til-',: :Txpr.,;v:vn.' of pr:)posltional 

hC" 16 .2.1 Coriiuii^: ^ it 1 .nal logic — truth 

UC^ •SO Truth 9Mo]y..ar^ 

iK'f init i.or. :)f a dvrivar. ion and us^•' of modus 
pon- rr^ * Am | 

hOL 22 C.)r..lit.i-Ml p7 oot pr ;i:e.rirc (CP) 

Working premir/^e (iVP) 
Roi.e oT i^r^br-lviiary i^;r .ivat^on:' 
UC5 1^ ?rd-ti r- viTj", -,ht;^ c^:indit1onal proof 

proceC.'^re 



'lOb 1 o^" !Infe>vnc*t? nii^X' 

Let t :'o^-.:\iN' t (:.:(:) 

UC ' '1 1 Of ■ T nf - r . s.'. i - ' - .. 

"'r^r^T!. . . dl^: ViH;- tiv.Ti ( F2) 
riV o J a:.v..t { il) 

1+06 ^8 J Tnr;.ir-'::v:t pro"':!: pr^: . .^'iurf flP) 

F r no I c^f r tZ^i^i ' 1" L^ v 

'Jo:rniuto the -'jnjuri;.:r,ion (CC) 
if09 P.*}':' fjonstrijctlng c Ar:^ ^rr^xamplcs by truth 

Fr'jb". •.:rri/. rn^qulre T^t/jdre-nt to deci'de whether 

f i^^r )--ur ir7a^ Lc.---crn3truct C'.)^in'oer* 
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Lesson 



Number of 
problems 



U09 
(cont. ) 

UlO 



''-^11 

J*15 
hlk 

^1-) 



hl6 



1+18 



ki9 

1+20 



29 



2Q 
30 

20 
52 



23 



29 



30 



19 
11 



miinber of 
hours 



Coritf.r.t. 



• 25 

• 87 



1.02 



1.10 



l,4ii 



1.15 
1.5^+ 



Elementary algebra: 

Well-i'urme.i formulas using equality ( = ) 
and inequality (>,<) relations 

Number definition (ND) 

Rules about equality: 
Commute equality (CS) 
Add equal term (AS) 

Rules about equality 

Subtract equal term (SE) 
Logical truth (LT) 

Review 

Replace equality (RE) 
More practice with RE 

Introduction to the INIT command to make 

up one's own problems 

Definition of "axlom"^ and '^Instance of an 
axiom" 

rirst axiom for a commutative group: 
Commute addition (CA) 

Second axiom for a ccmmutative group: 

Associative law (AS) 
Short forms of AS: 
Associate left (AL) 
Aj^;fociate right (AR) 

Remaining axioms for a commutative group: 
Zero (Z) 

Negative number (N) 
Additive inverse (Al) 

Theorems on addition 

Theorems 1-5 
Using theorems in a derivation 
Short forms for theorems and axlcms 

Theorems 5-7 

Axioms and theorems for a noncommutative. 

group 

Reprove theorems 1-7 without commutative 
axiom 
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(•Table I, cont.) 
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^ ^ Average 
Lesrson ^^^^ number of '^on^ent 



*''^0 F.xp: -nj-ur. tr.r t'l^.J ing-axloni? exercises 
)^;;; 59 Trv^iJ.i^la^i ' I'.rgiirii rcnt-riicea into firi3t- 

lit: ^titv-^/pretjitiori of a :^t nVence: 

ilr:v pi ;-/l.ic :ni^*.-!-prct.atlons in an 

Inti o":"-.'^'' i'..'^ ■un'ivc.r:7al and. existential 

1rar:oj-:^tl:.g Kii^ui/ri. ^^^T-nte ncer^ Into fir3t* 
r-rar'^r lor.!- '-J^'-^^'iv^C T^i^i^i^'-if i^r:^ 

1 n r -1 n : f • v ^^1 r ^ r- t r m 3 n t .■ 1 f i e r 23 1 
.M'i i. ^^.t -no rr.W on (KG ) 



^9 , .'/•.; ji. . i^:. of r- -r/.r i.. i -n;^ ruler: 

f^C'/'^ rr: ini^. r'en'. witii quantifiers 

q^:a, '-ru^, w'T' 



fe^^ensi'^n -^f :,nv.'j:'y :)f i orpr^nation to 

Prol)lom' 1^ Mi , ^.^r^^^nt clecld'??r> on 
validivy v:- u:i a:r\>2"^rrjrit in English 

Indep^.^ncfonct: mT r^viom:-- ( ippllca^ionL. of 
thovv y ■iri^.->"prer..a^ivjr 

i^jl 5I1 l,t5 Boolean alg^^r ra-^'''::i> nom':; 

(Jonjinute uni-:;-: v * 



(Table I, cont. ) 
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- Nvmiber of i ' * ^ ^ 

Lesson , , rvjmber t.ontent 

problems , 

^ hour" 



^,>0 S.^^^ Hrolean ol^-'olr:^ 

Duality 

Theorems I0I-I82 

1+53 16 Boolean algebra 

Axioms 



Subclass (SA) 
Theorems 185-19 



45*) U8 U.12 Symbollzation of English sentences 

especially related to the predicate 
calculus with identity; proof of 
equivalence of forms 
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Axiom and Theorem List for Lessons k0l-kS9 



Rules of Inference 

CON (P ->Q) IFF (NOT Q NOT P) 
DFA (P ->Q) IFF (NOT P OR Q) 
DNA NOT (P -♦Q) IFF (P Sr. NOT Q) 

FOR AIIY rcimilA S, 

QMA (A X) S(X) IFF NOT (E X) NOT S(x) 

CriB (A X) NOT S(X) IFF NOT (E X) S(X) 

QNC (E X) G(X) IPT NOT (A X) NOT S(X) 

CUD (E X) NOT 3(X) IFF NOT (A X) S(X) 



For the following axioms and theorems, assume universal quantification 

unlesf? otherwise specified. 



Axioms on Addition 

Ck (commutatlvity): 

AS (associativity): 

Z (zero axiom): 

N (negative number): 

AI (additive inverse): 

U (unity axiom): 



X + Y = Y + X 

(X + Y) + Z = X + (Y + Z) 

X + 0 = X 

X + (-Y) = X - Y 

X + (-X) = 0 

NOT 1=0 



Theorems on Addition 

THl: (-X) + X - 0 

TH2: 0 -f X - X 

TH3: X - X = 0 

THU: 0 - X = -X 

TH5: 0^-0 

TH6: X - 0 = X 

TH7: X + Y = X+ Z-* Y-Z 

TH8: X + Y = Z-» X = Z-Y 

TH9: X = Z- Y-» X+Y = Z 

THIO: X + Y = 0-» X = -Y 

THll: X = -Y-> X + Y = 0 

TH12: X+Y = X-» Y = 0 

TH13: -(-X) X 

TII14: (-(X + Y)) + Y - -X 

TH15: -(X + Y) = (-X) - Y 
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(Theoremn or. Addit ior.^ 





: (.X) - Y : 


■i) - 




Till'/ 


: - - y 






'mis 


0: - Y) " 7, 


\ + 






, - Y; - ■■ 


V _ 


(Y + :■;) 


Til. '0 


: X + (Y - X) - 


- Y 






: :•; - ;X + Y' ^ 


. -Y 




TI!.;- 


: CX - Y) 1 \Y 


- 5-0 


- X - z 



Ax.; t: ;- C vdcr 

j::;,T:;rotry : 
AH (additi'^n)- 
'I':- f "^rar:^ it lvi ty) : 
' ^:>r:^f^^t i vity ): 
W ^.ol'initioii >); 
ilL ( ,^ leiict r\unbor): 
1^1 (mo r;'eatGL;t number): 



X < Y NOT y < X 

X < Y X + Z < Y + Z 

X < y Y < z -> X < z 

NOT X - y X < Y OR Y < X 
X > Y IFF Y < X 
(A X)(E Y) Y < X 
(A X)(K Y) X < Y 



: .-"I.:; on Order 







Y 


NOT X = Y MOT Y < 


TII' 5 


V < 


0 


-> 0 < -X 




• 0 < 


_> 


X 0 




X + 


Y 


- ; X + z -> Y ' ' z 






Y 


~» -Y <- -X 


TI!- , 


- V - 




X X ' Y 




*' + 




Y) < X t (-Z) -) Z < Y 




/ < 


Y 


X + (-Y) < X + (-Z) 



A Useful Relation 
TH70: (A X} X < X + 5 

TH71: (A X)(A Y) X < Y + 5 OR Y < X + 5 

TH7/': (K X)fK Y)(K Z) X < Y + 5 fir Y < Z + 5 NOT X < Z + 5 



Boolean Al gebra Axioms and Theorems 

cu (a V H) - (H V a) 

UI (G V 0) - G 

nj (::• t h) (h t a) 

DU (G V (H t K)) = ((G V H) t (G v K)) 
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prove that the rect ot' ".i:e cta^ei;'c::tr., / / -^r^cvio ler:: v:u. from the 
axioms selected by use or the r* iles of Icvpca]. LSoi'ei.ce so far Intro- 
duced# The pro^j:rain itself determines whether the student has satisfac- 
torily conpleted ar. exercise, :^vid:ir.f^, a Cv>n)p].e^v. re^^oi't on tho axiorrii: 
selected and on which axioms and lemmas were needed by the student to 
prove a riven theorem. 

Two protlem formats are ijsed in the lessons on translating English 
into the formalism of the first-order predicate calculus. Lesson k22 
is restricted to iterative requests for possible translations until the 
student's response matches an instance of one of several stored correct 
answers. By Iiesson h^^, the student is expected to show a proficiency 
in determining whether or not two expressions are logically equivalent. 
Tlius^ if the student's symbolizatlon of an English sentence does not 
cor-espO!:d with one of those stored with the curriculum^ he or she must 
decide whether it is possible to prove logical equivalence. The student 
uses his skills in constructing proofs to show that an if-and-only-if 
relationship holds, or uses the method of interpretation to show that 
the equivalence :oes not hold. 

The system of inference for first- order predicate logic follows 
that of Suppes (1957) • The problems in Lesson k26 motivate the restric- 
tions on the use of the quantifier rules, and those In Lesson k2J have 
the student prove each of four q^iantlfier negation rules. A large num- 
ber of exercises at the end of each lesson give the students practice 
with the general principles introduced. 

The student receives a passing grade in the course if he completes 
Lessons hOl through k2S^ and does the first five finding-axioms exercises. 

10 
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Our intention was to iv.ahe availcrle set of optional lessons from which 
the students could select ones to v.se to qualify for & grade of B. Dur- 
ing the sprin*; aiid fall quarters, only one such set of lessons was avail- 
able: Lessons h^l throuf-:h -'03 on an axiomatization of Boolean algebra. 
Theorems for these lessons are also shown in Table II. Students desiring 
a grade of B did two additional finding-axioms exercises. lesson U3'), 
on s.vTnbolizing sentences in English, completed the course and the stu- 
der;t's requirement for a grade, of A. 

Tne course curriculum Is thus a linear sequence of lessons which 
the student follows. He can interrupt that sequence to make up his own 
problems, prove lemmas to help in proving problems presented to him, or 
move around in the course in a nonlinear fashion. This last feature 
proved beneficial when unplanned computer down-time meant that the stu- 
dent's history of problems completed was not properly recorded by the 
p:-o^2;ram. The student could skip ahead to the next problem in the se- 
quence without waiting for a proctor to patch hit. hir,tory file. On the 
average, the students used thli; program feature 11 times (12 times in 
the fall) out of an average of 62 sessions i 57 in tne fall) at the ter- 
minal. We found that thiz simple feature improved the students' atti- 
tude towards studying with an instructional system sometimes prone to 
electronic error. 

S ystem Usage 

A total of 179 students enrolled In the Stanford course during the 
four quarters fron fall, 1972, to fall, 1973, vith 121 students completing 
a grade of A, B, or pass. The distribution of students by quartar and 
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BEST copy AVAILABLE 



this report b'jr^OJ ..xi ' \^ v.^/^-t:), M Vv t. rj.::, --^k.;''/? ^.1 -,??r:, 
38 from the fail-19'f'5 class. 

■:\-:\tv^y;/ to the re-nlar pace of viaijy I'iiclures, students* work habits 
rope' ■ :::o^'v t-he "por-ic^k' c^'T'^v^.^i^ ■ oii^nrlered chara iterlstlc of rnO£5t research 
v.n'ke^T*. Alrno'^-t -^1 of them abct^iined from any contact with the course for at 
least -1 week;, [vot^itly durlnp; periods when pressures in other courses mounted* 
Dur'i:.^ 'he Bprin^r^ term, 1.1 stt.:der:ts tov^k a break at least one month. 

Ar> can \:e seen from Table TII^ the dropout rate was approximately 
lk/'> . ercerit, with a 3liKht bias upward in thi.? number because students 
iv; r .0 fall'-1975 class did not have any period except the Christmas vaca- 
t'l-y. :.o make up mcompleteii . Univer^uty regulations allow them to finish 
ti't* course within one year after enrollment # It is Interesting to com- 
pare this rate wltn two other cO'irses. The elementary Introduction to 
[.lri.l,\;opny^ which is a general course not oriented toward logic at all^ 
hci,- larger average enrollment this.n the logic course, and over the past 

years the average dropout rate has been IJ^^b per cento A course of a 
different sort, the first Intermediate course in logic, which is tech- 
nically harder and requires some mathematical and logical sophistication 
on the part of the students, has a smaller enrollment than the logic 
course by a factor of 3 or Summing the enrollment over the past 6 
years, the average dropout ra.te in that course has been 27*8 percent - 
Tt ^.'hu^ can be seen that, the aropout rate If more or less comparable to 
other courses. It should be emphasized that the University has a system 

1& 
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TABIJ; III 
Distribution of Grades 



BEST COnr AVAILABLE 







Grades at 


end 


of each quarter 




Quarter 










No 


Incom- 


Total 




a 


B 


Pass 


credit 


plete 


Fall IQ'^2 


15 


3 


9 


11+ 


13 


52 




3 


0 


) 




2 


12 


21 


Spring, 1975 


20 


0 




J, 


18 


1+6 


Fall, 1975 


22 


0 


11 


12 


1^ 


60 


Total 


58 


3 


28 


52 


58 


179 




Grades 


of 


incompletes 


finished as 


of January 


1 


, 197i+a 


Quarter 












No 


Total 




A 


B 




Pass 


credit 


Fall, 1972 


2 


i+ 






3 


1+ 


13 


Winter, 1972 


3 


1 






1+ 


1+ 


12 


Spring, 1975 


3 


3 






1+ 


8^ 


18 


Fall, 1973 


_2. 






-5 






Total 


10 


8 




11+ 


26 


1+8 



^According to University rules, students have one year after enroll 
ment to complete a course. 

^These students actually had further time available to complete the 
course . 
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BEST COPY m\\mi 

or eri-o: • • ■■ ' ^ ■ \ \ ' ' ' ' ■ X: 

courses a^.d ? ^^^^'i^'- ' ' .> . j.i.' ei.:- 

roll!T:ent, or..:.c ^'•^*y .lav-.- ir'^.ie ' or v^^-.j" ';'i^y v^ir: to t.al^e* 

(i|-5.8u iiOiirs)^ oonrlrie'l vnth onr es".imate of :/ime ir^pent off tne tei^unal 
worki:^-, -.^ : 'Iv^^'^^er prv^Mem;^ ca- tne f i.i:clin^^;-a>. 1 exercises (an esti- 
mate;; Vy l.in;:^.'; ^ ^■o^.jt":'^ -r >■:]r^^^!■:^ra^i^^-^• v-Mi r.ii'^^ five :mits of 
acadeiri'..' oro.l.: t; the t\ide':r'.vt recei''A-d. Tat)l^: 'i-V ^i-i-ve^L-^ rrie triearA and 

vai larce oi' r.inie :'pepJ_ on dll lesson?. TMs timing, Infomation is not 
!iorr!i3 : ; : t-/ ^he niimbTr of prot. iem:^ nc-r th^ problem typef.^, but does 
,i ;,he r-iaMve a if f v*lty in learnlnr" '-.-^rtain proof proceiiires 
Oi •.■:/;cr.>tr ta;i;:n* 1:: each ir^^^sDn. Thu:i we s-e tlrn-: increases In les- 
s.>rs 'r\ which tne student.^ ari t.auf^,nt indire-:^ proof procedure and in- 
'Ter^yre riil^e-^' f'^r q-.Kin^ i f i-^rt- Tw^: '.-rge vuri^^^^ie in Les^'on ^35 high- 
rh.iit'j the ;?hara^:^/?r o: ' :j iefb:^ on. eltnei a student cf%n easily translate 
che statemento t-xpresf^ed in tV^gllsh into tne predica'ce calculus, or he 
ha:.- to spend a sif^niricantlv longer time tryint^ different translations 
or proving i-hat niC: translation*^ jre are not lor.ically equivalent to 
the stored answers^ 

Another view of syc'tem ur-age war? compiled in order to examine each 
student *s deviation from the average amoant of time per leisson* Except 
for 9 Rt^jdents (8 fb^st^ 1 rlov), students were ^not uniformly fast or 
:^low with respect -"-o ■^rK- aver-a^V-* '-iiTi0.int of t-nr.e lesson. Moreover, 



TABLE IV g£5j jjQpy ny/yyjBLE 

Average Time per Lesson 



Lesson 


Average time 


standard 


No. of students^ 


in hours 


deviation 


conpleting lesson 


1+01 


.31 


.21 


4^1 




.21 


.05 


42 


i+03 


.50 


.37 


^3 


'+0i|. 


1.29 


.77 


45 


i+05 


3.12 


1.40 


45 


14-06 


1.56 


2.2o 


45 


U07 


1.00 


1.22 


45 


U08 


1+.1+3 


2.79 


43 


U09 


2.57 


.95 


43 


1+10 


.1+2 


.20 


1+3 


1+11 


.1+1+ 


oil 

.24 


^5 


1+12 


.53 


.2o 


43 


i+15 


.23 


.07 


li 0 


1+11+ 


.87 




45 


'+15 


1.01+ 


• 62 


43 


i+l6 


1.02 




li -x 

43 


1+17 


1.10 




^3 


1+18 


1.1+1+ 


.84 


1+5 


1+19 


1.13 




43 


1+20 


I.5I+ 


•95 


1+5 


1+21 


.1+0 


.20 


1+3 


1+22 


1.55 


• 71 


59 


1+23 


.1+8 


• 19 


1+0 


1+21+ 


1.65 


.61 


37 


1+25 


1.61 


• 60 


37 


1+26 


2.66 


lOl 


38 


1+27 


2.7^ 


l.Ol 


36 


1+28 


5.05 




35 



15 



(Table IV, cam.) 



Lesson 


Averagtj time 
in hour 11 


L'ituiKiar'd 
deviation 


TJ.-'. of students^ 
completing lesson 


k?Q 




5.30 


3^ 


U5I 


1.6? 




28 


•'+52 


^.56 


1.82 


25 




5 • 2*;? 


2.71 


25 




•V.12 




20 



Spring class only. 
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the students do r.o^. ipi^ear to slow dovni or speed up in a uniform manner, 
but are q\iite diverse in their deviation from the average. 

Summaries of computer usage, total uumter of sessions each student 
spent at the terminal, number of minutes spent, and tne number of central 
processor minutes each student's work require^^, are shown in Table V. 

Insert Table V about here 

Because the instructional cyst em is basically a proof checker, consider- 
ably more ccmputation time is required to carry out the computer- student 
interaction than is required by a traditional drill-and-practice system^ 

Students worked on Teletype (R) t^odel-35 terminals; terminals were 
available in a classroom at TMGGS, During the spring quarter, the stu- 
dents had access to 10 terminals frcm 5 ^0 p.m. until 6:00 a.m., Monday 
throu/^h I'Yiday, and sll day Saturd&y and Sunday. In the fall, the dally 
schedule was extended to allow all-day access during the week; Friday 
nights arid Saturdays' until u-00 p.m. were reserved for machine maintenance 
by the If-lSSS staff. 

A summary of the acxual times the students ufed the system is shown 
in terms of the total number of terminal hours per hour of the day in 
Table VI. Thus, the entry for hour '"^ Is the total number of terminal 

Insert Table VI about here 

hoiirs logged between midnight and 1:00 a.m. In the spring, 9:00 p.m. 
was the most popular hour for working; while 2.00 p.m. was preferred 

by the fall students, it wjc. i.ot ^.,valXd.ble to students in the sprinr.. 
In each case, terminals were mainly utilized during hovirs when there 

17 



Number of iZcvsiovj:- , :\-::nr:::t Time, «.:vjl T" Cycler 



— 




roT' J; 'a 


..... 


, ocring, 


1975 






student 
identif. 

number 


Number 
of 

sessions 


Minutes 
of connect 
titne 


mlnur.es 


Student 
identif. 


Number 
of 

sessi ons 




Minutes 
of connect 
time 


W 
minutes 


28-50 


9j 


2i+91..6'; 


122,6'? 


2875 


56 


3605.52 


110.35 


2831 




2019. '2 


74,65 


2677 


37 


1685.80 


70.00 


2852 




5508,72 


87..') 2 


2876 


53 


1857.92 


50.18 


28^; 3 


',2 


594^.8? 


105.37 


2879 


17 


2049 » 12 


71^.27 




24 


1556.97 


41J+5 


2880 


1+4 


5855.^^5 


106.87 




90 


5657.25 


125/92 


2881 


29 


1929.15 


I48.78 


28v6 


20 


2056 , 27 


96.53 


2882 


75 


5708.98 


110.52 


2857 


20 


U59O.77 


6s62 


2885 


91 


51+10.52 


121.28 


C8'38 


■So 


2777. 6 'j 


100 „ 78 


2384 


18 


1852.72 


29.53 




'■8 


2741.72 


92,57 


2866 


26 


2072.28 


76.85 


;-36c 


51 




ui , 53 


.■383 


17 


151+6.6'^ 


'+1.95 


2861 


49 


2581.52 


94.25 


2889 


67 


2758.68 


107.28 


2865 


5^" 


22i+8.00 


76.1^ 


2890 


55 


5285.87 


118.1+5 


2864 


6 5 


5747.55 


6'i.,78 


2091 


69 


2859.57 


92.97 


2865 


66 


5619-65 


105.10 


2892 


13 


807.95 


32.77 


2867 


88 


2661.62 


11 1 , 98 


2695 


288 


6710.22 


200.73 


2868 


17 . 


895.38 


24.78 


2894 


218 


I+U56.55 


II+I+.55 


2869 


57 


1512,75 


61+. 77 


2896 


27 


1700.72 


55.87 


2870 


?9 


2707.72 


87.67 


2897 


40 


I9I+O.I7 


62.70 


2871 




3018.16 


115.73 


2898 




2811.85 


69.1+2 


2872 
2875 


57 

^47 


1896.15 
2877.75 


56,67 
92.70 


Averages 


56.1+ 


271+2.52 


85.1+5 
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Number of Sessions, Connect Time, and CPU Cycles 
lor Each Student, Fall, 197 J 



Student 
identif. 


Number 
of 

0 C 0 0 4- i 9 


Mlnutss 
of connect 
time 


CPU 
minutes 


Student 
identif. 

number 


Number 

of 

sessions 


Minutes 
of connect 
time 


CPU 
minutes 


1301 


27 


2680.97 


75.55 


1555 


36 


2079*52 


64.42 


1502 




2357.02 


67.17 


155^ 


25 


2180. U7 




1.505 


'}2 


3791.35 


80. UO 


1535 




3297 -'^2 


95-28 


150U 


hi 


5518.07 


38,52 


1556 


37 


2^7^ -05 




1505 


38 


5J+25.22 


. 81.67 


1558 




1977.37 


55. oO 


1510 


28 


2070.32 


6l.i+2 


1559 


32 


2764.12 


95.07 


1311 


39 


2738.05 


90.68 


15^2 


1 1 

kk 


21+58.50 


69.83 


1512 


•52 


5013.20 


75.^8 


15^5 




3766.55 


9'^*hO 


1515 


30 


5877.25 


225 . 28 










131'+ 


33 


1855.22 


40,62 


15^7 


36 


26U7.22 


82.98 












27 


2220. UO 


H1k15 


1520 


31 


1766.35 


^+1.75 


15^9 


21 


1331.65 


55.23 


1522 


36 


1710.57 


71.58 


1352 


31+ 


1821.97 


39.55 


1323 




3335.15 


130.66 


1555 


15 


1216.63 


3^^.87 


1324 


54 


1857.67 


62.08 


1556 


35 


282U.07 


72.87 


1325 


li2 


2274.87 


65.35 


1558 


31 


IU36.I7 


1+4.67 


1328 


50 


2357.65 


65.25 


1559 


36 


m8.65 


57.78 


1550 


25 


2009.05 


59.28 


1560 


31 


1570.50 


63^05 


1531 
1332 


37 

i*3 


1855.17 
2506.30 


61.90 
60.08 


Averages 


36.9 


2422.61+ 


71.88 
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T 

Hours of 


Total nuiTibei' oi' 




t < s^* 1 ill 11 iCLX i 1 vJLi 1 


n 




1 












J+ 






1') 


6 


10 


7 


8 


8 


8 


9 


9 


10 


K; 


11 




11: 


> ( 


15 


9 


li* 


91 


lb 


It';.: 


16 


361 


17 




18 




19 


k'K 


20 


500 


21 


32Q 


22 


kQO 


23 





tablt: vTt 



Total rJumber of Hours Terminals Used 
at Each Time of Day (Fall, 19T3) 



Hours of 


Total number of 


the day 


terminal hours 


r. 




T 
X 




0 


Iri 


-3 








c 
>> 


Q 
✓ 


o 


g 




7 


o 
O 




0 






2C6 


IX 








15 




Ik 




15 


589 


16 


559 


17 


222 


18 


112 


19 


211 


20 


254 


21 


250 


22 


17^ 


?5 


lOU 
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war :\ . : . ^ ■ "' ' ■ .^^^ r.;..^ 

pop\:larity of Ihe .*'^;;:-ro : ■ - i ' - ": i / • -i:,..: !e . 

student -v:! I : 

thor>e G^.uvleiitP vh-o comr'lete'i LeCoon 'i/:') to iv^hk or^Jer all problem types 
a^v* M'd i to *:.hex'' proi^ereT.ce in doing the type of problem, with the ex- 
cep* rho CjTiVoll::iitlon problem^' found ii: IjeSGon 45';- Table VII 

Ins'-r^ Table VII about here 

shovr. that derivations unin^^ the rules of conditional proof (CP) and 
indirect proof (IP) were prefexred (also the niort frequently encountered), 
whil-'* ro-iterit ial derivar:or:r m ^^rneral ranked second. The lev rating 
-:u* p: "lems jriV^-^lvin^^ proof that axioin': are Independent or not, or prem- 
is^.-r ;-co:iS:i .stent or not, ir; alnioi't c^urely due zo the lov frequency of 
or '^;*r:'erce of these pro'b''-'?ri typei in the curriculum* Data on the stu- 
dent cir>ility to make ttit re^pjir'^.; olioices an these kinds of problems 
r.Iiov "^hn- th*^ ?^/ij ^n"]t7' intuitionr for determining consistency of prem- 
ises or ir.dependence of axioms were not a? well formed ac for determining 
the validity of an argument, 

V/e -i-lso asKed all the students to complete an attitude survey. 
Questions are shown In Appendix I; natingG on a 1-7 preference scale 
are /^iven in Table VIII. For the most part, students enjoyed the course, 

Insert Table VIII about here 

would like to take other c<.Tnputer-ba'">ed courses, and found they liked 

the active Interaction the r,yf?tem afforded. They were not happy If a 
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TABLF: VX: 

Student- Ranking of Preferred Problem Types 













Che: 












Problem type ^ 


1 




3 


k 






7 


8 


9 


10 


Mult/iple choice 








12 


5 


0 


0 


5 


5 


5 


Senten*: ial 
:i--rivati on 


5 


7 


■7 




y 




? 


2 


1 


5 


Counterexample by i 
truth assigment 


b 


h 




6 


C 




0 


2 




1 


Der iv.it i on^ i ng 
CP ana IP 


11 




r 


c 


5 


I 


5 


1 


1 


1 


Proofr. in 
elementary algebra 




5 


0 


2 


1+ 


h 


1 ■ 
*+ 


5 


2 


1 


Finding axioms 


h 


k 


1 


h 


5 


3 


6 


5 


2 




Derivations of 
formulas having 
quantifiers 


0 






1 


6 


8 


10 


1+ 


2 


0 


:'ount'?rexample by 
interpretation 


1 


2 


7 


1 


5 




7 


5 


4 


1 


Pr.)ving premie:^^'"- 
consistent or 
inconsietent 


2 




0 


1 


1 


k 


2 


7 


12 


5 


Proving axiomt^^ 
Independent or 
dependent 


2 


1 


0 


0 


2 


7 


5 


2 


5 


15 
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BEST COPY AVAILABLE Jesuit n or Que^t/ionmire i?.^ Studor.rc) 



on 
n-umber 


Gcale 


1 


2 


5 




i:; 


6 


7 


L 


7 


10 


1 


1 


1 


1 


5 


o 


19 


5 


1 


0 


0 


1 


0 




0 


C 


1 


2 


1+ 


5 


12 


u 


0 


0 


1 


1 


2 


2 


18 


r- 
J 


0 


0 


2 


0 


2 


k 


16 


o 


5 


5 


3 


4 


3 


5 


5 


*7 
( 


0 


5 




2 


5 




6 


Q 

o 


0 


0 


2 


6 


8 


1+ 


If 


9 


7 


5 


5 


1 


2 


2 


2 


10 


? 


5 


7.5 


0 


5 


5.5 


0 


11 


11 


3 




5 


2 


0 


1 


1^ 


0 




5 


6 


6 


5 


0 


15 


3 


6 


2 


2 


If 




5 


Ik 


7 


6 


6 


0 


1 


2 


2 


15 


5 


6 


6 


U 


2 


2 


1 
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human tutor vu- r.ot. n-^ll/^-l- -rr:i-r ■^V;d--^rB vere .^ isples sea 

that the tcrminaXs vvre not available bei'jr.- V.CC p.m. I^-- an outstand- 
ing majority, the Dt.'un'-ntE 1 ike i vorkinf.r it t.hr-ir ovn pace at tne ter- 
minals and enjoyed working/, on Uieir own^ Independent of the activities 
of their classmates . 

Based on cominentG maJe by th^^ ^-tudentr. on these questionnaires, 
several cominands were addea to the- GyGtem in order to facilitate reviev- 
inr. partial proof :i anc i^horrenm^^ solutions. Oo:nirent& by 3tuaents in 
the winter-l^?73 cias^: precipitated the aaditior. of the LE^rSCN comznand 
for Gklppirig around curricul'im problems. 

Example P roblems 

'The examples we hdva cr-.OGf?n to describe in detail wer« selected 
for one or more of the following rea:^onr.v (l) thti dlGtribution of num- 
ber of steps to complete a ^--olutlon indicated a gr^at variety of solv- 
tioni?, (2) rules required to solve th'j problem had a high percentage of 
errorG, and (j) svercig^.- r-jmber of hinT:s r»:i?,uesteu tor the problem showed 
that the problem wai: conr.idered dlffic;ilt by many '^^tr^dents. In each 
example presentation^ we provide a sample solution and the overall aver- 
age, minimum-, and maxim^arr. number of steps. Many problems are randomly 
selected. If a student stop's. a session in the middle of constructing 
a derivation, there is a good chance tnat he will not receive that same 
problem at his next ses'.ion^ Randomly asslgnea problems are numbered 
as: lesson.problem^l or lesscn#problem,2. 

A table of thf> minlminr. and ma^i'Tram nvmber of f^olutlon step-c? for all 
derivation problems ;o(jO j^uch problem:.; occur in the curriculum) is provided 



fall data. 

In the Jriit.lal iessoi.i-- o:' t.he coui'Se, irLiidentc learn to construot 
propositi oral or Gei.tent'ial ie.:- : v,vt:o".r; . They tfj-i^-ht a r ,::.;:le -'On;- 
iKand lanfiiuage consisting of mnemonics for inference rules, and conditional 
and indirect proof procedures. The s^Jininary of inference rules given to 
the (Students appears in Appendix '.'II. 

The first example, shown in Figure 1^ is interesting because it re- 
quires a subsidiary derivation whose hypothesis is Identical to one needed 

Insert Figure 1 about here 

In thi? main derivation- At the point of presentation of this problem, 
the r.tudent has learned the rules modus ponens (AA), working premise (WP), 
and conditional proof procedure (CT), 

Indirect proof .. The indirect proof procedure (IP) is taught in 
Lesson hO&, where the eighth problem^, shown in Figure 2, is the first 

Insert Figure 2 about here 

derivation example. To encourage the use of IP, the inference riole deny- 
ing the consequent of a conditional sentence is not permitted In solutions 
to any problems in this lesson. Control of this nature is part of the 
basic lesson driver. The tenth problem in the lesson, shown in Figure 5, 

Insert Figure 3 about here 

is significant because three hints were available^, and, on the average, 
students requested one hint. This was one of only ten problems in which 
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DERIVE 



(((R Q) ^Q) ((R Q) ->S)) ((R & Q) -» (Q 



WP (1) ((R ^ Q) -^Q.) -» ((R Q) -^S) 

W (2) R Q 

WP (5) £ 

W (H) R Q 

Tr3CF (5) (R Q) 

l.^AA (6) (R & Q) ->S 

^:2AA (7) S 

377CP (8) Q & S 

OcF (9) (R ^'^^ Q) & S> 

T79JP (10) ((R&Q) >Q) ((R Q) -»3)) -^{(R & Q) 



Average number of steps; 10.5 
Minimum number of steps: 10 
Maximum number of steps: 15 

Fig. 1. Sample derivation of Problem ii05,12.2 
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BEST COPY AVAILABLE 



DERIVE (NOT S) (3 NOT Q) 

]^ (1) NOT 5 

WP (2) 3 

i£ (5) 9l 

1. g.3IP (h) NOT Q 

2. h'CP (5) S -»NOT Q 

l.'jCP (b) (NOT 3) -4 (3 -» NOT Q) 



Average nmlDer of steps: 6.2 
Minimum number of steps: 6 
Maximum number of steps: 10 



Fig. 2. Sample derivation of Problem 408.8.2 
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BEST copy AVAILABLE 



DERIVK (NCT((W & R) -^S)) --^ (W ^i; R) 

HYP (1) WOT (iW 3e ~^e) 

WP (2) NOT (W R) 

WP (5) w & R 

W (k) NOT S 

g,3.^IP (5) S 

3.^ CP (6) {w Sc R) -^s 

ltt72IP (7) W & R 

i.7::p (8) (NOT ((w «c r) ->s)) (w & p) 



Average n'jmber 
Mininrnm number 
Maximum number 



of steps; 3.6 
of steps: 8 
of steps ; 1? 



Fig. 3. GfiLiiple derivation of Problem i+08.10.2 
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students seemed to hav(^ -.etTi'i -'-r.c.i r; l-lSsIoi. '-.it tnic t-rollem* The new 
rule used in the ^a-mple zolutioa is IIYl^ ro introduce the hypothesl'. ol' 
the desired conclusion. 

Truth values . Sxercires on aGr.igning truth values to atomic formu- 
las and on determining the truth val\ie of a complex formula prepare the 
student for the task of finding counterexamples to arguments that can be 
symbolized in the propositional calculus,. An example frt^m Lesson ^09^ 
seen in Figure shows the steps a student might take to decide on the 
validity of an argument. 

Insert Figure k about here 

Qca^' tabulation of problems in which the student had a choice be* 
tween constructing a counterexample or a derivation shows that this par- 
ticular problem was attempted 57 times,, 17 of which DER (derive) was 
correctly chosen first, 12 times as the second choice* The derivation 
is tricky because on^ premise is not needed in the solution- New rules 
used are f orm'-a-disjunction (FD) and commute-a-disjunction (CD), 

Identities > By Lesson Ull, the student constructs proofs using 
rules for identity: number definition (W), conimute- equals (CE), and 
addition of equal terms (AE), Distribution of steps in the example, 
shown in Figure 5, is especially scattered, possibly because the solu- 
tion can deceptively be constructed without a conditional proof. Only 

Insert Figure about here 
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DERm: NOT (R) 



P (1) Q, 

P (2) (S Sr Q.) ->NOT W 

P (5) W 

CONSTRUCT A DERIVATION OR FIND A COUNTEREXAMPLE 

TYPE DER OR CEX 

CEX 

PIND A OOUNTEREXAMPTE: 
NOT (R) . 
P (1) Q 

P (2) (S ^ Q) ^NOT W 

P (3) W 

YOU ARE LOCKING FOR A COUXITEREXAMPLE , THUS, THE 
CONCLUSION MUST BE FALSE AND THE PREMISES TRUE. 

NOW ASSIGN CONSISTENT TRUTH VALUES TO THE 

REMAINING SENTENCE LETTERS. 

S F 

W T 

Q T 

R f 

LET'S CHECK TO SEE IF YOUR TRUTH ASSIGNMENT 

MAKES THE PREMISES TRUE AND THE CONCLUSION FALSE. 

S & Q F 

NOT (W) F 

(S Q) -^NOT W T 

YOUR ASSIGNMENT MAKES ALL THE PREMISES TRUE. 

LET'S CHECK THE CONCLUSION. 

NOT (R) F 

CORRECT 



Average number of steps: 8.1 
Minimum number of steps: 7 
Maximum number of steps: 12 



Fig. h. Sample derivation of Problem k09,l6 
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BEST COPY AVAILABLE 



DERIVE 


(B = 


C ? 


f C = 


■ D) 




= B 




P 


(1) 






Sf C 


= B) 


D 


= B 


1CE2 


(2) 


(D 


= c 


."V B 


= c) 


-) D 


= B 


2CE1 


(5) 


(c 


- D 


B 


. c) 


-^D 


= B 




(M 


(B 


= C 


& C 






= B 



Averafee number of steps: 7*8 
Minimiom ntanber of steps: k 
Ntaximum number of steps: 17 

Fig. 5. Sample derivation of Problem lfll*ll^*2 
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one of the I'all sti!(ier:tr, but five spring, students, found the four-step 
solution. 

Replacement of a term by an equal term 'r^'' is introduced in Lesson 
kiy, more practice is ^■".iven wit.h this new r-ile in Lesson klk. Only one 
spring student and several fall stuaente found the clever eight-step 
solution in the example, shown in P'igui'e o. The proof requires the 
rule of number definition. 

Insert Fig^are 6 about here 

Axioms . Th*=' commutative axiom [Ck) is presented in Lesson J+15 and 
the associative law for addition in Lesson kl6. As o^jr analysis of 
rules shows, AR and AL ( ass oc lata right and left over addition) were 
difficult rules to use (as were the associative laws for Boolean algebra) 
•Two problems from Lesson klC are presenr.^H jn Figures 7 and 8. We 

Insert Figures 7 and 8 about here 

selected them also because they -.re typical of the symbolic manipulations 
performed in constructing proofs m eiomentary alj^.ebra cind in the propo- 
sitional calculus. The minimum proof in the second example from thin 
lesson does not require use of the AL rule, despite the fact that AL 
was Introduced immediately before presentation of the problem. Several 
examples of such improper ordering of problems— improper in the sense 
that their context is misleading— have been corrected in later versions 
of the course. 

Three more axioms are taught lu Leeson . zero (Z), negative 
number (N), and additive inverse (AI). The example fron this lesson is 



DERIVE 



NOT D < D 



p 


(1) 


D = 7 






p 


(2) 


6 = B 






p 


(5) 


5 + 1 




E 




(M 


6 = 5 


+ 


1 


27IfREl 


(5) 


5 + i 




B 


5.5AA 


(6) 


NOT 7 




B 


ICEl 


(7) 


7 = D 






0.7RE1 


(8) 


NOT D 


< 


B 



NOT 7 < B 



Average number of steps: 9'T- 
Minimum number of steps: 8 
Ifexlraum number of steps: 1^+ 



Fig. 6. Sample derivation of ?-oblem klk,7*2 
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DERIVE 8 = (2 + 1) + ^3 

ND8 (1) 8 - 7 + 1 

rai (2) 7 6 + 1 

1.2RE1 (:>) a = (6 + 1) + 1 
ndS (1^) = 5 + 1 

TT^REl (5) 8 - {(^ - 1) + 1) + 1 

(6) 6 - (5 + (1 + 1)) + 1 

ND2 (7) 2 = 1 + 1 

7CE1 (8) 1+1=2 

oTBrBI :9) 8 - (5 + 2) + 1 

9AR1 (10) 6 5 + (2 + 1) 

8CA1 (11) 8 --^ (2 4 1) + 5 



Average number of steps: 12,-5 
Minimum number of steps: 11 
Maximum number of steps: 17 

Fig. 7. Senpi'.e derivation of Problem i+ 16.19.2 
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DERIVE (5 + 1 = 1+ + 2) ->3 + (2+1) - 6 



HYP 


(1) 


5 + 1 = k + 


2 




(2) 


6 = + 1 




?.1RE1 


(3) 


6 = 1+ + 2 




3CE1 

mT 


ik) 


J+ + 2 = 6 




(5) 


= 5 + 1 




TTT^REl 


(6) 


(3 + 1) + 2 


= 6 


nAKl 


(7) 


5 + (1 + 2) 


= 6 




(8) 


5 + (2 + 1) 


= 6 


l.SCP 


(9) 


(5 ^ 1 = -f 2) 


■^3 



(2 + 1) = 6 



Average number of steps: 13.6 
Minimoni nianber of steps: 9 
Maximian number of steps: 25 



Fig. 8. Sample derivation of Problem 1+16.22.2 
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Interesting.: b,ecav,r,e the vide dlstr^Vnit ion of nvimber of steps might have 
resulted from the inability to use the short forms of the axioms. The 
short form for K (apply N in one step by letting the program determine 
the correct instantiation of the axiom) is explained Just before the 
example problem, shown in Fif::^ire 9, is assigned. 

Insert Figure 9 about here 

Theorems . Students first le^^^rn about theorems and their use in 
constructing proofs in Lesson ^l8. The example problem, shown in Figure 
10, is an extremely eimple one, yet only two students found the four-step 

Insert Figure 10 about here 

solution. They had just proved Tlieorem 1 ((-B) + B = O), and had been 
encouraged to use it in subsequent proofs. The problem is misplaced, 
but we are still surprised at the difficulties most students seemed to 
have. 

Lesson k20 is Included in tile curriculum to demonstrate a second 
axiomatic system, one in which the basic binary operation is noncommu- 
tative. We added this lesnon in the winter, 1975, because In our pre- 
vious expe^^ence we seemed to find students finishing with the impression 
that "the world commutes." Theorems proved in Lessons klQ and 419 must 
be proved in this new system; the proofs, without available, are of 
course more difficult than when CA is available. 

Problem 420. shown in Figure 11, is the proof of the first theorem. 
It is, in terms of the number of hints requested ".jy the students, the 

Insert Figure 11 about here 
57 



DERIVE C + (5 + (-C)) = 1 + (1 + 1) 

AI B + -B = 0 

B:C (1) C + -C = 0 

ND3 (2) 5 = 2 + 1 

ND2 (3) 2=1+1 

2.3RE1 (1+) 3 (1 + l) + 1 

itCA2 (5) 3 = 1 + (1 + 1) 

: C 4 -C (t) 3 + (" -C) = (1 + (1 + 1)) + (C + -C) 

6AI2 (■"'I 5 + (C + -C) ^ (1 + a + 1)) + 0 

7Z1 (8) 3 + (C + -0) - 1 + (1 + 1) 

(9) (3 + c) + -C = 1 + (.1 + 1) 

9aAl (10) {C ^ 5) + -C = 1 + (1 + 1) 

lOARl (11) C + (3 + -C) = 1 + (1 + l) 



Average number of steps* 12.6 
Minim-jm nimiber of steps: 11 
Maximum niaraber of steps ^ 20 



Fig. 9. Sample derivation of Problem ^17.24.1 
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BEST copy AVAILABLE 



DERIVE (0 + 6) + (11 - 5) = 6 4 (11 - (5 + 0)) 
Iff 



IZl 

2CA3 



Average nmber of steps: 10.5 
Minimum nmber of steps: k 
Maximijm nmber of steps: 5^ 



(1) 


(0 + 


6) + 


(11 - 


- (5 + 


0)) = 






(0 


+ 6) 


+ (11 


- (5 + 


0)) 


(2) 


(0 + 


6) + 


(11 ■ 


■ 5) = 




0)) 


(c 


+ 6) 


+ (11 


- (5 + 


(3) 


(0 + 


6) + 


(11 ■ 


■ 5) = 




0)) 




(6 


+ 0) 


+ (11 


- (5 + 


(M 


(0 + 


6) ^ 


(11 ■ 


- 5) = 








6 ^ 


^ (11 


- (5 H 


^ 0)) 





Fig. 10, Sample derivation of Problem 1+18.8.1 
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PROVE (-B) + B = 0 



AI B + -B = 0 
B:-B 



(1) (-B) + (-B) = 0 



Z B + 0 = B 



-B (2) -B + 0 = -B 



B:-B 
AI 
B:B 
5CB1 

; --B 

6AL1 



9Z1 
lOAIl 



(5) B + -B = 0 

(h) 0 = B + -B 

(5) (-B) + (B + -B) = -B 

(6) ((-B) (B -B)) + --B = (-B) + (-B) 

(7) ((-B + B) + -B) + — B =(-B)+ ("B) 

(8) (-B + BW ((-B)+ --B) =(-B)+ ^--B) 

(9) (-B + B) + 0 = (-B)4- (— B) 

(10) (-^+ B =(-B)+(--B) 

(11) (-:^ + B = 0 



Average number of steps: 15 • 8 
iMinimum nvimber of steps: 7 
Maximum number of steps: 5^ 



Fig. 11. Sample derivation of Problem k20.k.O 
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BEST COPY AVAILABLE 

most difficult problem in the curriculum. An average of approximately 
two hints were reqje oed, higher than any other problem. When this 
theorem was proved in "Ohe theory of Aoelian groups, the average number 
of steps for a solution was two, in contrast to the average of 13 steps 
required in proving the same theorem without the commutative axiom. 
This same difficulty held true in proving the remaining six theorems 
in the lesson, despite the fact that availability of Theorem 1 should 
have made the proofs as simple as when 'ZtX was available. 

Translation . v;e show only one example from Lesson k22 on translating 
English statements into the formalism of first-order logic (without quan- 
tifiers). It is a problem in which the student has already translated 
each premise and the conclusion as separate problems (Figure 12). The 

Insert Figure 12 about here 

sample proof uses De Morgan's T^aw (DM), def-inition of implications (DFA), 
deny-a-disjunct (W) , ana right conjunct (RC). 

Quantifiers . The notation we have developed for teletypewriter rep- 
resentation of '"for all X, S(x) ' and "there exists an x, S(x)" is 
"(A X)S(X)" and "(E X)S(X)", respectively. The commands for intro- 
ducing and eliminating quantifiers (Lesson k26) are universal specifi- 
cation (US), universal generalization (UG), existential specification 
(ES), and existential generalization (EG), Restrictions on the use of 
these COTBiands are taught in I-esson 427. 

The rules US and UG are used in the next example problem. Figure 15, 
in which the student shows the validity of an nrgimient about feather les? 

Insert Figure 13 ahout here 



Q = LOVE IS BLIND 

R = MEN ARE AWARE OF ^PHE FACT THAT LOVE IS BLIND 

S = WOMEN TAKE ADVANTAGE OF THE FACTT THAT LOVE IS BLIND 



DERIVE 


s 




P 


(1) 


(Q & NOT R) OR (Q & S) 


P 


(2) 


(NOT R) -^NGT Q 


2DFA 


(3) 


R OR NOT Q 


m 




NOT (NOT R & Q) 




(5) 


NOT (Q & NOT R) 


I.5DD 


(6) 


Q & S 


6RC 


(7) 


S 



Average mjunber of steps: 10. 5 
Minimijim number of steps: 7 
MaximiJin nmnber of steps; 20 

Fig* 12. Sample derivation of Problem 422. 39*0 
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ONLY BIRDS HAVE FEATHERS 

NO MAMMAL IS A BIRD 

EACH MAMMAL IS FEATHERLESS 

DERIVE (A X) (M(X) -»NCT F(X)) 

P (1) (A X) (F(X) -»D(X)) 

P (2) (A X) (M(X) -»NOT D(X)) 

lUS 

X:X (3) F(X) ^D(X) 
2US 

X:X (h) mix) -^NOT D;X) 

3(XN (5) NOT D(X) -^NOT F(X) 

T+ TIhS (6) M(X) -» NUT F(X) 

:X (7) (A X) (M(X) -»NOT F(X)) 



Average ninnber of steps: 9»1 
Minimum number of steps: 7 
Maximum number of steps: 17 



Fig. 13. Sample derivation of froblem k26Ak,0 
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•iftalnrialS. It is 'ITue'laEt problem in-t.-x- .=;.or:'t\.-;. I'he mlniiT:\;m seven-step 
proof uses the contrapositive rule (CON), one not especially emphasized 
in the curriculum. 

In the case of KC, we adopted the convents or. tliat the variable of 
quantification may only be replaced by an ' ambiguo^jis ' name. Ambiguous 
or arbitrary names are denoted by an asterisk followed by any variables 
occurring free in the formula to which ES is applied. 

The ride EG replaces all occurrences of a proper name or ambiguous 
name by a variable of quantification. We chose problem 55 from Lesson 
1^27 (Figure Ik) to demonstrate the use of the four quantifier rules. 

Insert Figure 1^* about here 

The problem is not necessarily a difficult one, making the wide distri- 
bution of the nianber of steps to soltition somewhat surprising. The fall 
students did much better than the spring students on this problem. 

In reviews requested during work on derivation problems, the pro- 
gram provides references to any flagged variables (i.e., those variables 
introduced free in a premise line) occurring in each line, as well as 
references to the premise line upon which the flagging depends. Again, 
this notation follows the rule usage adopted in Introduction to Logic 
(Suppes, 1957). 

Interpretation . The notion of interpretation of a set of state- 
ments written in English is introduced in Lesson 1+25 and then elaborated 
on in Lesson k2Q where the method for showing an argument invalid is ex- 
plained. We take the set of rational numbers as the domain of interpre- 
tation. To show that an argument is Invalid, the student provides an 



BEST COPY AVAILABLE 

DERIVE (E X) ^ (A Y) (P(Y) -NOT L(X,Y))) 

P (1) (A X) (P(X) ->H(X)) 

P (2) (E X) (I(X) & (A Y) m(Y) -.NOT L(X,Y))) 

2ES 

xTf (5) 1(0 ^ (A Y) (H(Y) -.NOT L(%Y)) 
lUS 

X:Y ih) P(Y) ^H(Y) 

3RC (5) (A Y) (H(Y) -.NOT L(*,Y)) 

^U3 

Y:Y (6) H(Y) -.NOT u{*,Y) 

k.ZnS (7) P(Y) -.NOT L(^,Y) 
TUG 

:Y (8) (A Y) (P(Y) -^NOT L(*,i)) 

>C (9) 

gTBFC (10) I(*) & (A Y) (P(Y) -.NOT L(%Y)) 
lOEG 

~ (11) (E X) (I(X) & (A Y) (P(Y) --.NOT L(X,Y))) 



Average number of steps: 12.8 
Minimvan number of steps: 11 
Maximum number of steps: 20 

Fig. Ih. Sample derivation of Pi'oblem ^4-27, 33-2 
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Interpretation of earih atomic I'ormv.l':: i i: * ::c 'irfroneiit av.j then proves 
the truth of an interpretation of a premise by deriving it as a theorem 
in the algebra of rational numbers. To prove a conclusion is false, 
the student derives the negation of the interpretation of the conclu- 
sion. 

The student facilitates the repeated need to prove such statements 
as (A X)(NOT X=X -» X=X) and (A X)(X*X X=X) by using the system's 
lUIT mode to prove lemmas. The ability Lo predict the usefulness of 
such lemmas shows a good level of understanding on the part of the better 
students in the course. We discuss this further in a later section. 

In addition to constructing derivations to show that an argument 
is valid or invalid, we use derivations to show that an axlcm Is depen- 
dent on other axioms or that a premise is Inconsistent with other prem- 
ises. We use an example from Lesson k29 (Figure 15 ) to demonstrate the 
format of such problem types. The students show that a premise is 

Insert Figure 15 about here 

inconsistent with other premises if its negation is derivable from the 
other premises. A set of premises is consistent if and only If It has 
at least one true interpretation (in terms of the axioms and theorems 
of elementary algebra). For example, students construct a proof that 
the two premises 

All unicorns are animals* 
Ho unicorns are animals, 
are consistent; in the figure, the student proves Axiom 1 is inconsistent 



he 



KST COnr WAIUBLE 



ALL MEN ARB MiWA:£ 
ALL ANIMALS ARE MOrf^AL 
SOME MEN AB£: NOT MORIAL 

SHOW THE THIRD PREMISE IS IN'X'N SI STENT 
WITH THE FIRST TWO PREMISES 

DERIVE NOT (E X) (.h(X^ NOT M(X)) 

(1) (A X) (H(X) -£(X)) 

(2) (A X) (S(X) -^M(X)) 

(3) H(X) -»SiX; 

(K) S(X) -»M(X) 

(5) KiX) -»M(X) 

(6/ NOT H(X) OR M(X) 

(7) NOT {H(X) NOT M(X)) 

(8) (A X) (NOT (H(X) & NOT M(X))) 

(9) NOT (E X) (H(X) & NO? M(X)) 

Average number of steps: 11.5 
Minimum number of steps:, 8 
Maximum number of "^zeps: l8 

Fjg, 15. Ssmple derivation of Problem ^+29.11.0 



P 
P 

lur 

X:X 
2UG 
X:X 
3AHS 

sdfa' 
o5m ~ 

:X 

8QN 



vith Axioms r and 5 ry coristr'jr'- ir^^r m rv'-o: , v^iiol: ♦^•^f^ first axiom Is 
false and the other two true. 

We selected one problem I'roin the lessons on Boolean algebra (l^'i^re 

16)* Tlie avera^^e munher oV hints for Uiln problem was one; two hints 

Insert Figure lb about here 

were actually available. This is proof of Iheoreiu 179* Data from the 
next three theorems also showed a high number of requests for hints. 

The symbolization problems in Lesson i^55 present another challenge 
to the students • intuitive reasoning. If the student's response is not 
logically equivalent to a stored answer, he mus^t construct a proof to 
show that logical equivalence does not hold between his answer and the 
one stored with the curriculum problem. He must also designate which 
of the two expressions will have the false interpretation and \rtiich the 
true, since the implication may hold only in one direction. To carry 
out proofs in this lesson, students were encouraged to prove (NOT NOT R) 
Itr which should have made problem ^55-13 (Figure 17\ an exercise in 

Insert Figure 17 about here 

proving two statements not logically equivalent, quite easy. Judging 
from the scattered distribution of solution steps, this suggestion did 
not seem to help. This problem was also the first time RQ, replace- 
equivalent-formulas, was available for use by the students. We show 
two solutions—one using quantifier rules, the other using RQ. 

Finding axioms . In a previous report (Ooldberg & Suppes, 1972) 
we examined the diversity of solutions obtained by students for the 

1*8 
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THEOREM 179 

PROVE (0 V H ^ a T h) -> G H 

HYP (1) G V H G t H 

TH168 a t (a' V H) G 
G: a 

H:H (2) G t (G V H) = G 

^.IREl (?) G t (G t H) G 

lA (G t H) T K - G t (H t K) 
G.-G 

H::a 

K:i{ IG T G) t H = G t (G t H) 

k .'5r£1 '■'^) fG t G) t H - G 

^jTnlo4.1 (t-j) G t K = G 

THV68 (G t (G V H)) - G 
G:H 

K:G (7) (H T (H V G)) = K 

lOUl (8) H V G = G t H 

TTBrEI (9) H t (G t H) = H 

qcn" (10) (G t H) t H = H 

lOIAl (11) G t (H t H) - H 

irmio^.l (12) G t H - H 

12.6RE1 ' (13) ' = H 

l,i;CF ilk) (G V H - G t H)-^G = ri 



Average nuinber of steps: 21,^ 
Minimum number of steps: 1^ 
Maximum number of step?- 38 



Fig. 16, Sample* derivation of Problem h23,Kh,0 
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• • • 

DERIVE NOT (A X) D(X) 



Proof 1: P (1) NOT (A X) NOT NOT D(X) 

'± (2) D(X) (STEPS 2-Q 

2DN (5) NOT NOT D(X) 

2.3CP (U) D{X) ^NOT NOT D(X) I£MMA.) 

W (5) NOT NOT D(X) 

5DN (6) dTxI 

^T^CP (7) NOT NOT D(X) -*D(X) 

TttlS (8) NOT NOT D(X) IFF D(X) 

iTBIqi (9) NOT (A X) D(X) 



Proof 2: P (l) NOT (A X) NOT NOT D(X) 

HYP (2) (A X) D(X) 

IQNC (5) (E X) NOT D(X) 

5ES 



X:* ik) NOT D(*) 

gus 

X:* (5) 

k.').21? (6) NOT (A X) D(X) 



Average nvnnber of steps: 9*8 

Minimum number of steps: 5 
Maximum number of steps: 19 



Fig. 17. Gauiple derivation of Problem U5U.I5.O 
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find inf?;- ax ions exercjses- The £tuaf-i.ts were seventh graders as well as 
collepie students. The data tor the classes analy?ed here were broadly 
similar, arJ cotiscquently we onrlt all uetaila. 

Choice of method * Three kinds of problems in the curriculum are 
designed to ^^ive the student an opportunity to develop some intuition 
about the validity or invalidity of nrK\iinents. I'hey Are (1) derive or 
find a counterexample usinp truth table analysir (DC), (2) derive or 
find a counterexample Tisinp; the method of interpretation (131), and (5) 
show premises inconsistent or give an interpretation to show premises 
inconsistent (DIC). In each case, the student states his choice by 
typing commands DEF. (derive), CEX (counterexample by truth analysis), 
or HIT (method of interpretation). He can change his choice at any 
time by typing one of the three choices; he can restart within the 
present choice by typing REo'IARI'. 

Table IX presents our data on the students' choices, sorted by 
problem number. It provides an enumeration of the nuF-ber of times each 

Insert Table IX about here 

problem was attempted, the correct choice-,, and the number of tinies the 
choice was correct on the first or the second try. Table X compiles 

Insert Table X about here 

the same data, sorting by problem type. There are 2'/ problems (1'3 with 
random selection) involved. Prom this information, it appears that the 
students have sane difficulty with each tyi..e of problem, some of which 
we feel was due to defeci^s in our presentation of uses of the method 
of interpretation. 



TABLE IXa 

Data on Students* Choices of Correct Method of Attack (Spring, 1973) 



Problem 
No, 


Problem 
type 


Correct 
choice 


IVT^ i fill T n 

NOf times 
chosen first 


JMO* ximes 

chosen second 


ioxcUt uiiiieD 
problem tried 




DC 


DliiR 


15 


0 






DC 


CEX 


10 


Q 
0 






DC 


DER 


10 








DC 


uhA 




*+ 


1 r 


409«1*^»1 


DC 




JLO 


*+ 




409«1h»£i 


DC 




ly 


c 
? 


51 


1|09*15*1 


DC 


CEX 




5 


oil 


409«15»2 


DC 


CEX 


lo 


u 


on 


409*16.1 


DC 


CEX 


19 


0 


of. 


1^09*16.2 


DC 


DER 


9 


4- 


1( 


1^09. 17*1 


DC 


CEX 


25 


d 




^4.09. 17. 2 


DC 


CEX 


11 


I. 
*f 


10 


1+09.18.0 


DC 


DER 


3 


0 


5 


1+09. 19*1 


DC 


CEX 


1^ 


5 




1*09. 19. c 


DC 


CEX 


5 




Ok 


i«.09.20.1 


DC 


DER 


9 


5 


lb 


409.20.2 


DC 


CEa 




7 


^9 


1+09.21,1 


DC 


CEX 


21 


1 


00 


1+09.21.2 


DC 


CEX 


lo 


0 




1+09.26.1 


DC 


DER 


^1 


5 


59 


1+09*26.2 


DC 


DER 


15 


-1 )i 

14 


55 


1+28. 17. c 


DI 


INT 


29 


10 


1+5 


1+28.18.1 


DI 


INT 


12 


1 


25 


1+28.18.2 


DI 


INT 


15 


1+ 


21 


1+28. 19.1 


DI 


INT 


9 


2 


13 


1+28.19.2 


DI 


DER 


12 


1 


26 


lf28.20.0 


DI 


INT 


1. 


3 


16 


1+28.21.1 


DI 


DER 


20 


1 


23 


1+28.21.2 


DI 


DER 


7 


0 


9 



52 



BEST cm miMi 

(Table IXa, cont.) 



Problem 
No. 


Problem 
type 


Correct 
choice 


JMO* biizies 
chosen first 


chosen second 


1 oi^cix X imc b 

protlem tried 




DI 


DER 


13 


u 


1? 




DI 


DER 


*t T 

17 


u 




42c5*25#l 


DI 






Q 


1 Q 




DI 


llMi 


XI. 






^4'2o«2^«0 


DI 


INT 


11 






42o^25.0 


TNT 

DI 


IKT 


1 k 


c 




h /^D Oil" 

42o.2d.O 


DI 




1 k 


VJ 


1 7 
X f 


429.20.0 


DI 


INT 


50 


16 


50 


If 29. 21.0 


DI 


INT 


52 


22 


86 


429.22.0 


DI 


INT 


53 


10 


55 


429.12.0 


Die 


INT 


28 


12 


46 


1+29.13.0 


Die 


DER 


28 


5 


41 


429.14.0 


DIG 


INT 


18 


20 


39 
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TABU'' LXb 



ijata on 


Students ' 


Choices 


of Jorrect Method 


of Attack (Fall 


1973) 


Problan 
No. 


Problem 

type 


Correct 
choice 


No. times 
chosen first 


No. times 

chosen second 


Total times 
problem tried 


1*09.12.1 


DC 


DER 


15 


8 


21 


1*09.12.2 


DC 


CEX 


8 


8 


18 


1*09.13.1 


DC 


DER 


10 


8 


22 


1*09.15.2 


DC 


CEX 


11 


6 


18 


1*09.11*. 1 


DC 


CEX 


11* 


7 


22 


1*09.1^.2 


DC 


CEX 


15 


2 


18 


1+09.15.1 


DC 


CEX 


15 


6 


21 


1+09.15.2 


DC 


CEX 


15 


2 


16 


1+09. 16 -1 


DC 


CEX 


17 


3 


20 


1*09.16.2 


DC 


DER 


8 


8 


20 


1+09.17.1 


DC 


CEX 


15 


1* 


19 


09.17-2 


DC 


CEX 


11* 


2 


16 


1*09.18.0 


DC 


DER 


1 


0 


3 


1+09.19.1 


DC 


CEX 


18 


1 


22 


1+09.19.2 


DC 


CEX 


1* 


1 


18 


1*09.20.1 


DC 


DER 


17 


5 


22 


1*09.20.2 


DC 


CEX 


17 


1* 


21* 


1*09.21.1 


DC 


CEX 


21* 


0 


25 


1*09.21.2 


DC 


CEX 


13 


1 


11* 


1*09.26.1 


DC 


DER 


26 


12 


1*1* 


1*09.26.2 


DC , 


DER 


17 


13 


36 


1*28. lY.O 


DI 


INT 


27 


16 


hi 


1+28.18.1 


DI 


INT 


16 


2 


20 


1*28.18.2 


DI 


INT 


11 


9 


21 


1*28.19.1 


DI 


INT 


16 


6 


22 


1*28.19.2 


DI 


DER 


7 


0 


28 


1*28.20.0 


DI 


INT 


17 


0 


20 


1*28.21.1 


DI 


DER 


22 


0 


27 


1*28.21.2 


DI 


DER 


ll* 


2 


19 



51* 



(Table DCb, cont.) 



Problem 
No. 


Problem 
type 


Correct 
choice 


No. times 
chosen first 


No. times 
chosen second 


Total times 
problem tried 


U28.22.I 


Dl 


DER 


20 


1 


22 


428.22.2 


DI 


DER 


15 


0 


18 


U28.23.I 


DI 


INT 


11 


7 


23 


^1.28.25.2 


DI 


INT 


19 


3 


25 


1^28.2^^.0 


DI 


INT 


8 


2 


17 


1^28.25.0 


DI 


INT 


9 


5 


15 


1+28.26.0 


DI 


DER 


12 


3 


18 


1+29.20.0 


DI 


INT 


33 


14 


58 


1+29.21.0 


DI 


INT 




15 


6k 


1+29.22.0 


D' 


INT 


57 


9 


h9 


1+29.12.0 




INT 


28 


16 


52 


1+29.15.0 


LC 


DER 


1+1 


5 


55 


I+29.II+.O 


DIG 


INT 


15 


23 


hi 



55 



TABIil Xa 

Choice of Method of Attack by Problem Type (Spring, 1975) 



No. of first i first Wo. of sec- ^ second* No. of 

Problem choices choice ond choices choice times type 

correct correct correct correct occurs 

DC 522 6^.5 51 5^ ^ 

DI 516 . 6k,Q 87 50.9 W 

Die 7^^ 58.7 57 71.2 126 



*G:lvei first choice incorrect. 
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TABLE Xb 

Choice of Method of Attack by Problem Type (Fall, 1975) 



Problem 
type 



No. of first % first No« of sec- 
choices choice ond choices 
correct correct correct 



b second* No. of 
choice times type 
correct occurs 



-DC— 
DI 

Die 



-290- 
357 



65.9 



62.9 



511 
151+ 



♦Given first choice incorrect. 
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Special Coirnnarids 

Several special featui-es of the instructioTial system are of Interest: 
hints, reviews, initiative, and redoing solutions. In the previous sec- 
tion we displayed three of the ten problems in which the average number 
of hints requested was approximately one, or, in one case only, two. 
Students reported that they did not always ask for a hint when they 
needed one (seeking a teaching assistant Instead) because they were not 
aware they were able to do so or, when they did, no hints were available. 
We found that the students in the spring quarter had a 57 percent error 
rate with the HINT commandj 1,291 out of 2,232 uses of HINT were handled 
with the comment "NO HINT AVAIIABLE." In the fall, the error rate for 
INIT was 5U percent, that is, 1,32U out of 2,UU9 uses of the ccimnand. 
Detailed data on all problems which either had a stored hint or for which 
at least one student requested a hint are shown in Table XI. An extended 

Insert Table XI about here 

discussion of the problem of helping students with hints and other methods 
is given in Goldberg (1975). 

A simple visual aid, reprinting the partial solutions with error 
messages and deleted lines omitted, was provided when the student typed 
the command RE'/IEW. REVIEW was used on the average about kh times in 
the spring quarter and about 69 times in the fall quarter. Several 
students were recorded as requesting a review of their work more than 
100 times. In other cases, students never used the conmand* This 
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TABLE XTa 

Data on Availability and Use of Hints (Spring, 1973) 



Problem 
no. 



No. of 
hints 
available 



No. of 
hints 
requested 



UOU.18.1 



0 



Uoi^.ro.i 

40U.22.1 
U0U.22.J? 
UO5.2.I 
1*05.2.2 

U05.3.I 

^05.5.2 

lf05.U.l 

.U05.i+.2 

"1*05.5.1 

1*05.7.1 

1*05.7.2 

1*05.8.1 

1*05.8.2 

1*05.10.1 

1*05.10.2 

1*05.11.1 

'105.11.2 

1*05-12.1 

1*05.12.2 

1*06.1*. 2 

1*06.18.2 

1*07.1.0 

1*07.1*.! 

1*07. 11*. 1 

1*07.15.2 

1*07.20.1 

1*07.21.1 

1*07.21.2 

1*08.6.0 

1*08.7.1 

1*08.8.1 

lf08.8.2 

1+08.9.1 

1*08.9.2 

1*08.10.1 

1*08.10.2 

1*08.11.1 

1*08.11.2 



0 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

3 

1 
1 



3 
10 
12 

1 
1 

2 

I* 

15 
k 

8 
11 
2 
6 
7 
7 
1 

3 

5 

6 

1* 
1 
1 
1 
1 
1 
1 
2 
5 
5 

11* 
2 
18 
15 
5 
6 

68 

^3 
10 
10 



Problem 
no. 



No. of 
hints 
available 



No. of 

hints 
requested 



1*08.12.1 
1*08.12.2- 

1+08.13.1 
J+08.13.2 
l*C8.11*.l 
1*08. 1I+. 2 
J+08.15.1 
1*08.15.2 
1*08.16.2 

1*08.17.1 
1*08.17.2 
1*08.18.1 
1*08.18.2 

1*08.19.1 

1+08.20.0 

1*08.21.0 

1*08.22.0 

1*08.23.1 

1*08.25.2 

l+08.2»+.i 

)i08.2l*.2 

1*08.25.1 

1+08.25.2 

1*08.26.1 

1*08.26.2 

1*08.27.0 

l+OS.28.0 

U09.I.O 

1^09.12.1 

1*09.13.1 
1+09.16.2 

1*09. £0.1- 

1*09.21*. 1 

1*09.21*. 2 

1*09.25.1 
1*09.25.2 
1*09.26.1 
1*09.26.2 
1*10.16.1 
1+10.16.2 



1 

1 
1 
1 
0 
1 
1 
0 
0 
0 
0 
0 

1 
1 

0 
2 
1 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
G 
1 
1 
0 
0 
0 
0 
0 
0 



6 

6 
0 
11 

5 

12 
16 
1* 
1 

2 

3 
1 
10 

7 
1* 

17 
1* 

15 

5 
5 
8 

2 

15 
11 
1* 
2 
1 
2 
1 
1 

3 
1* 

11 
33 

15 
1* 

9 
2 
1* 
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(Table XIa, cont.) 





No. of 


Ko. of 


Problem 
no* 


No. of 


No. of 




hints 


hints 


hints 


hints 




available 


requested 


available 


requested 


410.17.2 


0 


1 


41S.18.1 


0 


1 


1+10.26.2 


0 


1 


4i8.19.1 


0 


1 


1+10.28.1 


0 


3 


418.21.2 


0 


1 


i+io.c'y.L' 


0 


1 


418,25.1 


0 




1+11.14.2 


0 


2 


418.25.2 


0 


3 


412.24.1 


0 


1 


419.3.2 


0 


1 


412.24.2 


0 


1 


419.11.2 


0 


1 


412.27-2 


0 


1 


419.13.1 


0 


1 


414.2,1 


0 


1 


420.4.0 


0 


95 


414.2.2 


0 


1 


420.5.0 


0 


21 


4li+.5.1 


0 


1 


420.6.0 


0 


1 


414.8.2 


0 


1 


420.8.0 


0 


2 


414.12.1 


c 


1 


420.9.0 


0 


11 


414.14.2 


0 


1 


420.10.0 


1 


32 


415.21.2 


0 


6 


422.19.0 


0 


2 


415.24.2 


0 


1 


422.23.0 


0 


1 


415.25.1 


0 


1 


422.27.0 


0 


1 


415.25.2 


0 


6 


422.28.0 


0 


2 


415. 26'. i 


0 


5 


422.39.0 


0 


2 


415.26.2 


0 


2 


424.11.0 


0 


5 


415.51.2 


0 


2 


424.12.0 


0 


1 


41S.52.2 


0 


1 


426.4.0 


0 


1 


416.8.1 


0 


1 


^26.6.0 


0 


2 


416.6.2 


0 


1 


426.9.1 


0 


1 


416.9^1 


0 


3 


426.11.1 


0 


1 


416.9-2 


0 


3 


42a, 11. 2 


0 


2 


416.14.0 


1 


0 


426.32.2 


0 


2 


416,18.2 


0 


2 


426.38.2 


0 


5 


416.19.1 


0 


1 


427.31.0 


0 


4 


416.19.2 


0 


1 


427.32.1 


0 


2 


417-17.1 


0 


1 


427.32.2 


0 


5 


417-18.2 


0 


1 


427.33.1 


0 


2 


417.19.2 


0 


1 


427.33.2 


0 


1 


417.25.1 


0 


2 


427.35.0 


2 


0 


417.25.2 


0 


1 


427.36.0 


3 


17 


417.26.2 


0 


1 


427.37.0 


1 


0 


r 417.27.1 


1 


10 


427.38.0 


1 


2 


418.3.0 


0 


2 


427.40.0 


1 


5 


418.8.1 


0 


2 


427.42.0 


0 


1 


418.8.2 


0 


2 


427.49.0 


0 


8 


^- - 418.10.2 


0 


1 


428.13.0 


0 


29 


t^'-"y 418.14.1 


0 


2 


428.14.0 


0 


3 


418.17.1 


0 


4 


428.15.0 


0 


2 
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(Table XIa, cont. ) 





No. of 


No. of 




No. of 


No. of 


irFODXcIU 


hints 


hints 


hints 


hints 


no* 


available 


requested 




available 


requested 


1^28.17.0 


0 


1 


452.25.0 


1 


15 


J+28.20.0 


0 


2 


452.27.0 


1 


53 


U23.23.1 


0 


2 


452.28.0 


0 


16 




0— 


1 — 


452. 50. 0 


5 


45 


429.12.0 


0 


4 


452.55.0 


0 


8 


429.15.0 


0 


2 


452.54.0 


0 


3 


429.14.0 


0 


2 


452.57.0 


2 




429.17.0 


0 


5 


452.58.0 


5 


45 


429.18.0 


1 


6 


452.41.0 


0 


10 


429.20.0 


1 


21 


452.42.0 


0 


2 


429.21.0 


2 


72 


452.45.0 


0 


2 


429.22.0 


0 


56 


452.44.0 


2 


42 


451.2.0 


0 


1 


452.45.0 


2 


39 


451.7.0 


0 


5 


452.46.0 


2 


74 


451.9.0 


0 


5 


452.47.0 


2 


55 


451.17.0 


0 


3 


if55.5.0 


0 


1 


451.20.0 


1 


1 


453.6.0 


0 


2 


451.21.0 


0 


2 


455.7.0 


0 


14 


451.25.0 


0 


2 


455.8.0 


2 


51 


451.26.0 


0 


6 


455. 9.0 


1 


25 


451.55.0 


0 


3 


455.10.0 


1 


30 


451.54.0 


0 


11 


455.12.0 


1 


33 


452.10.0 


0 


6 


455.15.0 


1 


12 


452.12.0 


0 


3 


455.14.0 


1 


27 


452.15.0 


0 


1 


455.11.0 


0 


2 


452.14.0 


0 


3 


455. 12.0 


0 


2 


452.16.0 


2 


15 


455.15.0 


0 


25 


452.21,0 


1 


51 


455.16.0 


0 


12 


452.24.0 


1 


26 


455.17,0 


0 


9 
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Data on Availability and Use of Hints (Fall^ 1975) 



Problem 
no. 



No. of 

hints 
available 



No. of 

hints 
requested 





Q. 


2~ 


— i»oi».i;.p 






404.22.1 


2 


5 




2 


15 


405.2.1 


0 


5 


405.2.2 


0 


1 


405.4.1 


0 


6 


405.4.2 


0 


6 


405.5.2 


0 


1 


405.8.1 


0 


9 


405.8.2 


0 


1 


405.9.1 


0 


5 


405.9. 


0 


3 


405.11.2 


0 


1 


405.12.1 


0 


2 


405. 1-. 2 


0 


5 


406.10.1 


0 


1 


407.5.1 


0 


1 


407.21.2 


0 


5 


408.6.0 


0 


14 


40B.e.l 


0 


9 


408.8.2 


0 


10 


408.9.1 


0 


2 


408.9.2 


0 


5 


408.10.1 


5 


3U 


408.10.2 


5 


52 


408.11.1 


1 


11 


408.11.2 


1 


1 


408.12.1 


1 


7 


408.12.2 


1 


3 


408.13.1 


1 


7 


408.13.2 


1 


6 


408.14.1 


1 


9 


408.14.2 


0 


7 


408.15.1 


1 


16 


408.15.2 


1 


16 


408.16.1 


0 


4 


408.16.2 


0 


1 


408.17.1 


0 


2 


408.17.2 


0 


2 


408.18.1 


0 


1 


408.18.2 


0 


2 



Problem 
no. 



No. of 

hints 
available 



No. of 

hints 
requested 



408.19.1 



408.19.2 
408.20.0 
408.22.0 

408.23.1 

408.25.2 

408.24.1 

408.24.2 

408.25.1 

408.25.2 

408.26.1 

408.26.2 

408.27.0 

408.28.0 

409.1.0 

409.12.1 

409.18.0 

409.20.1 

409.22.0 

409.24.1 

409.24.2 

409.25.1 
409.25.2 

409.26.1 

409.26.2 

410.16.1 

410.16.2 

410.22.2 

410.27.2 

411.11.2 

411.24.2 

412.22.2 

412.24.2 

414.2.2 

414.18.1 

415.21.2 

415.25.2 

.415.26.1 

415.26.2 

415.27.1 

415.32.1 



0 
1 
2 
1 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

1 
1 

0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 



2 
0 

12 

5 

17 

1 

7 
3 
3 

8 

12 

3 
2 
2 
2 
1 

5 
1 

8 

13 
21 

13 
9 

10 

3 
1 
1 

5 
1 
1 
1 
2 
2 
1 
1 
4 
8 

3 
1 

1 



ERIC 
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(Table XTb, cont.^ 



Problem 
no. 



No. of 

hint- ■ 
available 



No. ot" 
hints 
reque^tea 



Ul6.a.2 

416.9.1 
1+16-.9.2 



Ul6.19'l 

i+i7/i"a 
417.17.2 

4:7^2sa 

^17 , dtj .. 2 

417.27.; 

j^i8.e.,2 

4 18. 10 ,.2 

i.ie..iU.2 

U18.17.1 

418. 23.1 

4l8.2^.2 

413.27.1 

420,4.0 

420.5^0 

420.10,0 

J*22.i"^.C 
422.26.0 
422.';9.0 

^2i+.:.i ,0 

U24.12,C 
42t.3.0 

426, t.O 
U2t'.9.1 
426, 11 .1 
426.52.2 
426.58.2 
426.44.0 
^27.51.0 
427.52.1 
427.52.2 

427.55.1 
427,55.2 
427.55.C> 
427.36.0 
427.57,0 

427. >6X 
427.40.0 



0 
0 
0 



0 
0 
0 
0 

0 
0 

1 
c 
c 
c 

0 

c 
c- 

0 
0 
0 

1 

0 
0 
(.' 
0 
0 
0 
0 
0 
0 
0 
(5 
0 

0 
o 

c 

0 
0 
0 
2 
3 

1 



1 

X 



■3 
1 

2 
2 

1 

150 
18 
52 
2 
1 
4 
4 

1 
1 



3 
1 
3 
1 
6 
5 

1 

3 
1 
0 

9 
0 

0 



Problem 
no. 



No. of 
hints 
available 



No. of 
hint? 
requested 



427.49.0 
428oI5.0 
^28.15.0 
428.1^.0 



428-19-1 

428. ^-: ,2 
428.22.1 

428,.C2„2 

4;>8,2_5»:. 

1*28,2^,0 
4>'8.26',0 
429,6.0 

429, ':i.o 

429,12.0 
429.15^0 
1x29.1^-0 
429.17'0 
29, 18 .0 
u29,2C''„0 
^■-•9 "21.0 
429.22.0 

451. r„o 
.9-0 

.20,.C 
.21,0 
,?4,0 
i.51„.i5_,C 
45:. -.26.0 
4;' .,^4,0 
i^52.8,C 
u52.1C«0 
452. 12.0 

4 52. 16:, C 

45c'.21,C 
452,24.0 
452.25.0 

452. 27. C 

45P.26.O 
452,50.0 
452.35.0 

4 ')<^.'jh.O 
452.57.0 



4*1 
451 



0 
0 

0 

Sl 



0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

1 

1 
2 
0 
0 
0 
0 

1 

0 
0 
0 
0 
0 
0 
0 
0 
2 

1 
1 
1 
1 

0 

3 
0 

0 
2 



5 

18 

3 

2 



1 
1 
2 
1 
1 
3 
1 
1 
6 
1 
5 

5 
4 

3 

6 

55 
54 

55 

2 

1 
2 
0 
1 
1 
5 

12 
1 
4 

17 

26 
24 

15 
40 
13 
69 
9 
7 

56 



ERIC 



65 



(Table Xlb, c.\it.> 



__ , . No. of No. oi 
Problem , . , ... 

hints hints 

available requested 


Problem j^^^^^ ^^^^^^ 

available available 


432.38.0 3 '52 
452.41.0 0 12 

432.42.0 0 1 


433.8.0 ? ^5 
433.9.0 1 52 
455.10.0 1 38 


43g.43.0 0 g 

452.44.0 2 66 
'!5-.4',.0 2 48 
432.4^-^.0 2 43 
452.47.0 2 83 

433. ^;.o 0 1 
433.6.0 0 1 
43^.7.0 0 k 


—433.12.0 1 — 32 

433.13.0 1 14 
433.14.0 1 31 
455.11.0 0 2 

455.13.0 0 17 
455.16.0 0 9 
455.17.0 0 6 
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InforrnatlO!! Is tabvilated In Table XTT. Another vay to get a fresh start 

Insert Table XII about here 

on a solution is to delete all lines except premises (which are, of 
course, part of the problem «?tatemen"Cy . Out of 7,577 (8,251 in the 
fal l) uses of thfe me D LL ( delete the la&L lines), £,017 (£,601) w e r e 
requests to restart a proof— about 27 percent (52 percent in the fail). 
So this was a generally useful command. 

The third feature is under curriculum control. It is possible to 
require certain commands to be used, or not used^ in the construction 
of a proof. If the student canpletes a solution that is incorrect 
solely because it does not meet the constraints, he is asked to redr 
the problem. This is utilized in the curriculum to 

(1) get the student to use a newly taught rule or recently proved 
theorem; 

(2) encourage the student to try diverse solutions; and 

(5) encourage use of short forms of rules by constraining the use 
of replace-equals rule (RE), 

Students in both quarters had to redo a solution only about 3 times 
out of 35 possible problems that included such constraints; only h stu- 
dents avoided the need to redo a problem. 

Many students had difficulty finding a solution within the con- 
straints of problem '+15* 30 (ostensibly chosen to encourage the use of 
the short form of CA.): 

USE LT IN THIS PROBLEM. 
DERIVE 5+6«6-^5 
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TABLE Xlla 

Gtiident Use of Sp';?cial Commands 
(Data by Individual Students, Sprinr, 1975) 



Student 


NEWS 


INIT 


LESSON 


REVIEW 


REDO 


2850 


15 


75 


49 


28 


1 


2851 


2 


6 


0 


0 


If 


285? 


9 


10 


0 


lk5 


2 


2855 


2 


31 


18 


89 


k 


285U 


1 


8 


0 


35 


2 


2855 


8 


25 


29 


9k 


2 


2856 


5 


60 


7 


165 


7 


2857 


8 


20 


5 


32 


3 


2858 


5 


22 


37 


79 


2 


2859 


7 


27 


20 


Ik- 


2 


2860 


2 


8 


3 


12 


0 


2861 


6 


2lf 


22 


7 


2 


286: 


9 


6 


0 


8k 


3 


286ii 


5 


llf 


2k 


67 


k 


2865 


k 


10 


7 


0 


3 


^^867 


5 


26 


15 


61 


1 


2868 


k 


7 


1 


65 


3 


2869 


5 


7 


0 


89 


7 


2870 


7 


30 


19 


0 


0 


2871 


k 


13 


15 


9 


2 


2872 


20 


h5 


17 


80 


3 


2873 


3 


20 


1 


k6 


3 


2875 


6 


21* 


18 


15 


2 


2877 


2 


11 


3 


73 


k 


2878 


5 


11 


0 


0 


5 


2879 


9 


31 


10 


26 


1 


2880 


1 


18 


Ik 


59 




2881 


7 


Ik 


2 


5 


5 


2882 


1 


30 


13 


i^3 


2 


2885 


12 


33 


9 


0 


3 




4 


5 


0 


0 


2 


2886 


w 


w 




0 




2888 


1 


15 


0 


21 


2 


2889 


20 


35- 


37 


61 


3 


2890 


0 


21 


19 


23 


1 


2891 


6 


22 


12 


0 


2 


289''-^ 




1 


0 


0 


3 


2895 


12 


30 


0 


15^* 


8 


289^+ 


5 


17 


1 


29 


2 


2896 


5 


17 




33 


If 


2897 ' 


10 


16 


0 


0 


5 


2898 


5 


Ik 


21 


79 


2 


Totals 


2U9 


865 


k62 


1822 


122 


Average 


5*9 


20.6 


11.0 


k3*k 


2. 
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Student Use of Special Commands 
(Data by Individual Students, Pall, 1975) 



Student 


NEWS 


IKIT 


LESSON 


REPAIEW 


REDO 


1501 


22 


32 


11+ 


217 


3 


1302 


8 


6 


0 


56 


1+ 


1305 


11 


13 


17 


27 


5 


1301+ 


6 


16 


7 


196 


2 


1305 


10 


53 


29 


39 


7 


1310 


7 


25 


19 


1 


3 


1311 


6 


22 


6 


27 


0 


1312 


k 


56 


21 


120 


6 


1315 


k 


20 


6 


137 


1+ 


131^ 


7 


5 


0 


52 


2 


1316 


16 


61 


22 


189 


2 


1320 


5 


9 


11+ 


67 


5 


1322 


k 


27 


25 


68 


1+ 


1323 


7 


1+1 


18 


1+6 


1+ 


1321+ 


5 


22 


16 


^1 


1+ 


1325 


7 


28 


1 


28 


2 


1328 


20 


56 




90 


1+ 


1330 


k 


15 


7 


31 


2 


1331 


7 


17 


1 


2 


1 


1332 


7 


6 


6 




1+ 


1333 


7 


21+ 


5 


3 


2 


135^ 


9 


ii+ 


6 


1+1 


2 


1335 


9 


16 


19 


63 


0 


1338 


25 


2h 


2 


89 


1 




D 














J J 


IS 


228 


0 


13^+6 


2 


11+ 


16 


35 


1+ 


13^+7 


17 


15 


12 


61+ 


1+ 


13^8 


1+ 


9 


2 


38 


6 


1349 


11+ 


51 


1+9 


3 


1+ 


1352 


1 


22 


11+ 


53 


1+ 


1355 


9 


11+ 


5 


20 


1 


1356 


9 


22 


11+ 


107 


1+ 


1358 


8 


8 


2 


30 


1+ 


1359 


12 


18 


9 


271 


3 


1360 


15 


29 


11 


0 


2 


Totals 


328 


81+0 


1+61 


2691+ 


115 


Average 


9.1 


23.3 


12.8 


71+. 8 


3 



« 
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An example chosei. to '^vr..-c=w]'d.':£-' rHv^: rtr^^ :• lo:, nn.tht -'n 'i-lB.;?'^); 

CONSTRUCT A DERiyATlCI] miOli DOES NOT UHK CA, 
YOU WILL HAVE TO USE AS AND ND. 
DERIVE >f2=2+5* 

It Is, frankly, not a very cleverly chosen prol^lerri; It is pai't of the 
planned revision to replace lt# 

Students received information about seminars and system schedules 
by typing NEWS, This command was infrequently used, an average of approxi- 
mately 6 times (8 in the fall) in contrast to approximately 62 (57) average 
sessions at the terminal • Teaching assistants preferred to list and 
post the course news because printing news at the lO-character-per-second 
outpvvt rate was a time-consuming task for each student to undertake. 

We have already mentioned the relatively large percentage of times 
LES3CTI W8S used to alter the sequence of problem presentations. Our scan 
of the protocol data indicates the command was used to skip ahead the ma- 
jority or the times it was called upon, LESSON represents one of the 
commands students could type after obtaining the initiative (INIT command) 
to request their own problems. Another command was FA (to select a 
finding-axioms exercise). Two other commands, DERIVE and PROVE, are 
analyzed in a subsequent section. 

Use of the Ccxnmand lan^age 

In order to carry out solutions to the curriculum problems, the 
students must learn to use a new language: a set of commands having 
a strict syntax and a semantics corresponding to that of the first-order 
predicate, calculus with identity* Example problems demonstrated the use 
of this language. Each command has, at most, four parts: 
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BEST copy AtfAtUBl£ 

(1) a list of references to previous lines of the derivation, 

(2) a comnand name, 

(5) a list of references to occurrences of terms or formulas, and 
(k) requests for terms and formulas* 
for example, we have (student input underlined): 
1.2,3IP indirect proof procedure, requiring 5 line 

references; 

1AR2 associate right over addition^ requiring 

a line reference and a reference to the 
occurrence of an instance of (B-»-C)+Dj 

IFD form a disjunction, requiring a line ref- 

erence and a request for a well-formed 
formula; 

lES existential specification, requiring a 

request for an ambiguous n'-jne; 

lEG existential generalization, requiring both 

the ambiguous name and the variable of 
generalization; 

NG (A X)(E Y)(X<Y) an axiom, requiring instantiation of all 

universally quantified variables ^oee scope 
of quantification is the entire formula; 

1TH3.2 short-form notation for a theorem (all 

theorem names are of the form TH followed 
by a number), requiring a line reference, 
theorem name, a,nd reference to the occurrence 
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tiie r.lieorem; and 
WP (i) R OH Q working premise, the only conimand that 

expects the st-udent to ^.vie the actual llrie 
of the proof. 

Heveral kinds of error messages coiild be sent to a student using 
a command improptrly. Syntax-error messages described the correct num- 
ber of line or occurrence references required, or stated if a formula 
or term was not well formed. Application-error messages commented on 
(a) attempts to use a command not learned -yet or not proved; (b) inability 
to locate the referent of an occurrence number; (c) attempts to refer to 
a line that is no longer available because it is part of a completed sub- 
sidiary derivation; (d) attempts to use a line as a working premise which 
was not introduced as such; and (e) improper applications of the quanti- 
fier :'-ileG iWf ^IG^ ES, EG; according to the restrictions taught in 
LessoiiS and k27. 

Table XIII Itemizes each name entered in the ccamnand language by 
the curriculum author, the total number of times students used the com- 
mand, and the number and percentage of times errors were noted in the 

Insert Table XIII about here 

attempted use of the command, The^e errors are further broken down 
into the kinds of errors: syntax or application. 

If we view the curriculum as providing lessons on and practice with 
manipulations of formulas in symbolic logic, the table presents no sur- 
prises. (Further support for taking this point of view is offered in 
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TABLE xiiia BEST COPY AVAILABLE 

Data on Rule Usage Summed over Students and Exercises (Spring, 1975) 



nuie 


No. of times 


Wo. of times 


Syntax 


Appl. 


$ total 


rule used 


error in use 


errors 


errors 


errors 


DFA 






0 




1 1 

XX 


DM 






d. 




x;) 


OON 




10 


5 


5 




AA 


9201 


911 


151 




Q 


CC 


loO 


57 




X4 


5ti 

^x 


CD 


575 


loO 


T ). T 
141 


39 


XI 


CE 


5055 


^40 




kP 






Q 


4 


0 


C. 




AE 


1905 


OU 


04 




k 
4 


SE 


925 




n c 


z: 
D 




DIFF 


0 


U 


r\ 
U 




A 

\J 


DC 


1050 


10 5 




1 kk 
144 


xo 


DD 


2145 




J\ 






DM 




115 






7 


DN 


1450 


4U 


4U 


A 
U 


0 
ez 


DS 


/- 


2 


i. 


■1 

X 




FC 


26 04 


-1 


99 


X 


0 


FD 


I50O 


Qc 
05 


Ac 


r\ 
\J 




HS 




23 


5 


OA 


f 


LB 


105 


27 


5 


cicL 




LC 


4151 


174 


^2 


1 kO 
X4£i 


k 

4 


LT 


45IO 


219 




xo 


k 
4 


RC 


5925 


147 




IX f 


0 


QNA 


169 


56 


2 


54 


01 

2X 


QNB 


4O9 


17 


4 


1 z 

15 


k 
4 


QNC 


216 


21 


2 


T A 

19 


9 


QND 


424 


22 


5 


17 


5 


CA 






72 


c n 

5 f 




A 

AS 


650 


51 


20 


25 


■7 
( 


Z 


1635 


151 


50 


1 01 
X2X 


9 


N 


1490 


140 


21 




9 


AI 


1195 


119 


22 


97 


9 


U 


115 


5 


5 


0 


1* 


NS 




25 


9 


Xh 


1* 


AD 


8 


1 


0 


1 


12 


TR 


iSh 


12 


u 


8 


7 


CN 


13U 


5 


u 


1 


5 


DG 


116 




10 


14 


20 


NL 


567 


5 


5 


2 


0 


m 


472 


9 


5 


if 


1 


AR 


1519 


265 


18 


2if7 


17 


AL 




154 


16 


138 


20 
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(Table XTTia, :ront..) 





No. of times 


No. of timecs 


Syntax 


Appl. 


€ total 


rule used 


errcr in use 


errors 


errors 


errors 


UI 




Od. 


15 


kO 
49 


A 
0 


cu 




■jlO. 

3o 


1^. 


26 




DI 


o21 


13d 


54 




xo 


DU 


699 


05 


19 


DO 


T 0 

Xfi 


II 






1^ 
10 


XI 

51 


f 


lik 










c 


EM 






Ik 
X4 




7 

f 


UA. 


27 


It) 


T 

X 


9 


37 


UR 


152 


ok 




01 




UL 


00 


14 


1 


15 


15 




170 




1 0 


Q 


XX 


CS 


161 


Q 


5 


5 


4 


lA 


241 


oft 


Q 
0 


on 


XX 


Till 


c Q r\ 

5oO 


4o 


25 




Q 


■.?}I2 


605 


42 






0 


TII5 


293 


22 


5 


17 


f 


rriTT 1» 


195 


15 


k 
4 


XX 


7 

f 


TH5 


254- 


5 


0 


X 


X 


TII6 


9o 


3 


T 
X 




J5 


TIi7 


60 


17 


3 


X4 


oft 


TH3 


2 


0 


0 


f\ 
0 


u 


TH9 


1 


0 


0 


0 


0 


TiIj-0 


0 


u 


U 


\J 


u 


Tilll 


0 


0 


0 


r\ 
U 


u 


. Till 2 


0 


0 


0 


0 


U 




4* 

1 


0 


0 


0 






1 


0 


0 


0 






0 


0 


0 


0 




THI6 


5 


0 


0 


0 


U 


THI7 


1 


1 


U 


X 


xuu 


THlo 


0 


0 


0 


0 


0 


xH19 


0 


u 


r\ 
\J 


U 


u 


TH20 , 


0 


0 


0 


0 


0 


TH21 


2 


1 


1 


0 


50 


TH22 


1 


0 


0 


0 


0 


th6o 


10 


0 


0 


0 


0 


^61 


161 


29 


26 


X 


18 


TH62 


1U8 


5 


2 


3 


5 


TH63 


2 


0 


0 


0 


0 




1 


0 


0 


0 


0 


TH65 


1 


0 


c 


0 


0 


TH66 


5 


1 


1 


0 


20 


TH67 


If 


1 


1 


0 


25 


TH68 


0 


0 


0 


0 


0 


TH69 


2 


0 


0 


0 


0 
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(Table Xllla, cont,) 



BEST COPY mum 



Rule 


No. of times 


No. of times 


Syntax 


Appl. 


JK total 


rule used 


error in use 


errors 


errors 


errors 


THfO 


11 


c. 


T 
X 


X 


0 


TH7I 


55 


1 


X 


A 


X 


TH72 






n 


X 


X 


iiiXO-L 


yo 


k 


p 


2 


k 




79 






J? 


> 






11 


X 

0 


A 
w 






^0 




(i 
\j 




7 

f 


THIO5 


220 


OCT 
25 


0 


xy 


XXf 


THlbo 


275 




Q 

9 


h 


L 


THlDf 


155 


( 


u 


7 
f 


h 


THloo 


199 




Q 


Xw 




THlo9 


lo 


0 


0 


A 


1 P 

Xl^ 


THI70 


lo 


D 


0 


A 




THi71 


13 


f\ 
\J 


U 


A 


A 
U 


THI72 


1 


1 




A 


Ik 

X*f 


TH1T5 


70 


10 


9 


X 


1 p 




122 


-LT 


x^ 






THI75 


7 


1 


1 

X 


A 


1 k 

XH- 


THlfO 


7 


T 

X 


X 


A 


Tk 

X*T 


THI77 




5 


0 

2 




0 


TH170 


37 


5 


0 


X 


Q 
0 


THI79 


21 


2 


2 


A 


A 

9 


THlbO 


^5 




fi 
0 


p 


pp 


t III Tt 0 T 

THlol 


55 


cr 

5 


J 






THlo2 


13 


£L 


X 


X 


1? 


THIO5 


1? 


A 
U 


A 
U 


A 
\J 


A 
\J 


fnvT^ Oil 


1 


r\ 
U 




A 


A 


THI05 


2 


u 


U 


A 


A 


THI06 


1 


0 


A 
U 


A 


A 


THI87 


72 


7 


5 


2 


A 

9 


THloo 




7 


0 


X 




THI09 


7 






A 


Ik 


TIII9O 


12 


1 


1 


A 

U 


0 
0 


THI9I 


10 


2 


2 


A 


OA 


THI92 


5 


0 


0 


0 


0 


THI95 


0 


0 


0 


0 


0 


CP 


13035 


1209 


517 


892 


9 


DLL 


7577 


270 


2U3 


27 


3 


EG 


2092 


200 


8 


192 


9 


SS 




857 


69 


788 


25 


mm 


2232 


1291 


0 


1291 


57 


HYP 


79^7 


0 


0 


0 


0 


IP 




7V2 


176 


566 


17 


m 


29^8 


70 


60 


10 


2 
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(Table Xllla, con-:,.) 



Rule 


No. of times 
ruxe usea 


No. of times 
error in use 


Syntax 


Appl. 


fl) total 

Ox X WX 0 


RE 


8821+ 


816 


190 


626 


9 


REVIEW 


1778 


0 


0 


0 


0 


RQ 


68 


14 


10 


4 


20 


UG 


3146 


195 


47 


148 


6 


US 


kk21 


502 


47 


255 


6 


WP 


14702 


15 


15 


0 


0 



74 



TABLE Xlllb 

Data on Rule Usage Sianmed over Students and Exercises (Fall, 1975) 



Rule 


No, of times 
rule used 


Ko, or times 
error In use 


errors 




^ \i\J wClX 
ex X UX D 


DFA 


1032 


111 


8 


103 


10 


DNA 


650 


58 


1 


57 


9 


CON 


275 


7 


1 


6 


2 


AA 


9911 


1018 


224 


794 


10 


CC 


175 


U5 


3^ 


11 


25 


CD 


516 


129 


103 


26 


25 


CE 


5^53 


223 


204 


19 


4 


OQ 


28 


7 


3 


4 


25 


AE 


1765 


66 


58 


8 


3 


3E 




53 


29 


4 


3 


DIFF 


0 


0 


0 


0 


0 


DC 


IO5U 


185 


29 


156 


17 


W 


2363 


335 


56 


277 


14 


DM 


1951 


13^+ 


19 


115 


6 


DN 


1610 


ko 


40 


0 


2 


DS 


13 


8 


8 


0 


61 


FC 


3081 


157 


157 


0 


5 




1516 


62 


62 


0 


4- 


HS 




29 


4 


25 


9 


LB 


152 


23 


6 


17 


15 


LC 


U7Jt2 


163 


31 


132 


3 


LT 


i+602 


172 


160 


12 


5 


EC 




186 


40 


• 146 


4 


QJIA, 


?32 


i+5 


2 


45 


19 


QNB 


816 


36 


10 


26 


4 


QNO 


258 


15 


h 


11 


5 


QND 


979 


26 


5 


21 


2 


CA 


27^5 

1 y 




80 


75 


5 


AS 


619 


b7 


20 


^1 


10 


Z 


1552 


115 


25 


88 


8 


N 




108 


20 


88 


8 


AI 


1035 


91 


16 


75 


8 


U 


251 


11 


11 


0 


4 


NS 


552 


14 


11 


3 


2 


AD 


37 


1 


1 


0 


2 


TR 


163 


8 


5 


3 


4 


CN 


124 


6 


1 


5 


4 


DG 


i;8 


13 


8 


5 


9 


NL 


75^^ 


13 


12 


1 


1 


NG 


561 


10 


9 


1 


1 


AR 


1233 


207 


2t 


181 


16 


AL 


677 


142 


8 


154 


20 
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(Table Xlllb, cont.) 


Rule 


No. of times 


No. of times 


Syntax 


Appl. 


Ttt total 


rule used 


error in use 


errors 


errors 


errors 


UI 


713 


60 


11 




8 ■ 


cu 


1030 




10 


3I* 


1+ 


DI 


762 


71 


25 




9 


DU 


612 


65 


20 


i*5 


10 


II 


J+91 


3i* 


9 


25 


5 


RA 


679 


58 


9 


29 


5 


EM 


650 


36 


8 


28 


5 


TIA 


15 


6 


0 


6 


1*0 


UR 


210 


32 


8 


2V 


15 


UL 


115 




3 




29 


SA 


11*3 


10 


6 


h 


6 


CS 


151* 


7 


7 


0 


J* 


lA 


230 


39 


6 


33 


16 


THl 


528 


ifO 


21 


19 


7 


TH2 


575 


52 


15 


37 


9 




192 


9 


6 


3 


1* 




177 


12 


2 


10 


6 


THS 


198 


7 


7 


0 


3 


TH6 


9^* 


3 


1 


2 


3 


TH7 


58 


15 


If 


11 


25 


TH8 


5 


0 


0 


0 


0 


TH9 


1 


0 


0 


0 


0 


THIO 


0 


0 


0 


0 


0 


THll 


1 


0 


0 


0 


0 


TH12 


0 


0 


0 


0 


0 






0 


0 


0 


0 




0 


0 


0 


0 


0 


THIS 


0 


0 


p 


0 


0 


THI6 


0 


0 


0 


0 


0 


THI7 


1 


0 


0 


0 


0 


THI8 


0 


0 


0 


0 


0 


THI9 


0 


0 


0 


0 


0 


TH20 


1 


0 


0 


0 


0 


TH21 


2 


1 


1 


0 


50 


TH22 


2 


0 


0 


0 


0 


TH6O 


1 


0 


0 


0 


0 


THol 




u 


u 


u 


u 


TH62 


212 


6 


6 


0 


2 


TH65 


h 


1 


1 


0 


25 


WSk 


1 


0 


0 


0 


0 


TH65 


3 


0 


0 


0 


0 


TH66 


3 


1 


1 


0 


33 


TH67 


0 


0 


0 


0 


0 


TH68 


2 


0 


0 


0 


0 


TH69 


7 


0 


0 


0 


0 
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(Table Xlllb, cont.) 





No. of times 


No. of times 


Syntax 


Appl. 


56 total 


rule used 


error in use 


errors 


errors 


errors 


TH70 


99 


1 


1 


0 


1 


TH71 




1 


1 


0 


1 


TH72 


5if 


0 


0 


0 


0 


THI6I 


l6i+ 


6 


5 


3 


3 


TH162 


93 


2 


1 


1 


2 


THl6^ 


167 


10 


1 


9 


5 




230 


6 


2 


U 


2 


THI65 




11* 


5 


9 


6 


THI66 


291 


11 


1 


10 


3 


THI67 


158 




7 


8 


9 


THI68 


268 


18 


5 


15 


6 


THI6Q 


16 


5 


1+ 


1 


31 


TH170 


10 


0 


0 


0 


0 


71 


Q 

y 


0 


0 


0 


0 






2 


2 


0 


50 


THl 7^ 


72 


18 


15 


5 


25 


TH17i4. 


150 


7 


6 


1 


5 


rplJT 7c 


2 


0 


0 


0 


0 


TH17f^ 


2 


0 


0 


0 


0 






1 


0 


1 


2 


TH1 7P 


2Q 


0 


0 


0 


0 


TWl 7Q 


X 


0 


0 


0 


0 


THI8O 




1 


0 


1 


1 


TH181 


U2 


7 


1 


6 


16 


TH1A2 


s 


0 


0 


0 


0 


THlft^ 


1 


0 


0 


0 


0 


xriXL/*T 


2 


0 


0 


0 


0 


Till ftc; 


1 

X 


0 


0 


0 


0 


IXlXOC? 




1 


0 


1 


35 


injLo f 


^7 


6 


6 


0 


6 






6 




1 


15 


THI8O 


19 


6 


6 


0 


31 




18 

xw 


2 


2 


0 


11 




16 


0 


0 


0 


0 


TH192 




1 


1 


0 


33 


TH195 


0 


0 


0 




r\ 
U 


CP 


12729 


1660 


J+47 


1213 


13 


DLL 


8251 


110 


83 


27 


1 


EG 


2568 


249 


6 


243 


10 


ES 


^715 


1138 


84 


1054 


24 


HINT 


2Wf9 


1321^ 


0 


1324 


54 


mcp 


9750 


0 


0 


0 


0 


IP 


1*917 


788 


174 


614 


16 


m 


2750 


■98 


95 


5 


3 
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(Table XITTb, cont.) 



Rule 


No. of times 
rule used 


No. of times 
error in use 


Syntax 
errors 


Appl. 
errors 


% total 
errors 


RE 


8856 


78U 


183 


601 


8 


REVIEW 


2728 


0 


0 


0 


0 


RQ 


122 


50 


3:? 


15 


KO 


UG 


51^91 


25U 


61 


193 


7 


US 


5^92 


ko6 


70 


336 


7 


WP 


15626 


2? 


23 


0 


0 
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the next section.) The percentage of errors In frequently used senten- 
tial rules was low except for IP (indirect proof procedure). Tlie IP 
errors are application errors, usually seen as the inability to select 
the two 13nes containing formulas that negate one another, or, if properly 
selected, to order the three line references. Undoubtedly, the percentage 
of errors for IP would decrease if the program were slightly less strict 
on the order of line rererences (unless order were essential, as it is 
in forming a conjunction). This is a simple programming change. 

Rules for commuting a disjunction or conjunction (CD and CC) also 
showed high error rates. If the difficulty were in determining whether 
a formula is a conjunction, disjunction, or implication, the error would 
show as an application errcr. Rather they are syntax errors, possibly 
omitting or incorrectly referring to the occurrence of the logical 
connective. 

Application errors in using commands that regroup parentheses (AS, 
AR, AL, UR, UL, UR) are also exceptionally high. These are difficult 
rules to use because they demand recognition of patterns or axiom sche- 
mata in which grouping is important, and require the ability to count 
occurrence-i (in a left-to-right manner) of these patterns in a formula. 
AR and AL are short forms for AS; similarly, UL and UR are short forms 
for US. Since the number of errors in using short forms of other axioms 
(Table XIV) is not as high, and those other axioms have simpler structures, 

Insert Teble XIV about here 

we can conclude that extra depth of parenthetical structure is a source 
of difficulty. 
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VhlilE XTVa 

i3ata on Use of Giiort Forms of Rules Summed over 
Students and Exercises (Spring, 1975) 



Rule 








exilic? 0 UsCLL 






CA 


3165 


• 

2i+21 


7 


AS 


650 


hk 


0 


Z 


1655 


728 


9 


N 


1I+9O 


959 


1+ 


AI 


.1195 


505 


8 


U 


113 


0 


0 


NS 


36k 


li^O 


0 


AD 


8 


1 


0 


TR 


16k 


11+ 


0 


CI-I 




16 


0 


DO 


116 


51 


0 


NL 


567 


0 


0 


no 


1+72 


0 


0 


UI 


77U 


.262 


2 


cu 


985 


957 


12 


DI 


821 


k06 


10 


DU 


699 


kk9 


9 


II 


280 


k 


RA 




203 


3 


EI^. 


685 


255 


7 


UA 


27 


12 


0 


SA 


178 


39 


0 


CS 


161 


71 


3 


lA 


2U1 


k2 


0 


THl 


580 


201 


3 


TH2 


.603 ■ 


27k 


.0 


TH3 


293 


120 


1 


THU 


195 


50 


' 0 


TH5 

THo 


25^ 


105 


0 


98 


22 


0 


TH7 


60 


55 


0 


TH8 


2 


0 


0 


TH9 


1 


0 


0 


THl 3 


1 


0 


0 


THlif 


1 


0 


0 


THI6 


5 


0 


0 


THI7 


1 


0 


0 


rH21 


2 


0 


0 


TH22 


1 


0 


0 
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(Tabled XlVa, cent.) 



Rule 


No. or 


No* as 


JiiTTOT 111 


times uoed 


srior* I orms 


Sfior U X OxmS 


TllbO 


10 


0 


0 


Tirol 


161 


0 


0 


Tlib-'? 


l!f8 




0 




2 


0 


0 




1 


0 


0 


•nib' 5 


1 


0 


0 


•rH66 




0 


0 


THoT 




0 


0 


Ttl69 




0 


0 




77 


0 


0 


T":'7i 


55 


0 


0 


TK72 


61 


0 


0 


T;Il6l 


96 


15 


0 


TKI62 


79 


k 


0 


T:a65 


199 


96 


0 


THloif 


266 


152 




'.niib'-; 


228 


81 


6 


TH166 


275 


92 


5 


'niio? 


153 




0 




199 


?o 


0 


':7ii69 


16 




0 


Till 70 


18 


2 


0 


11(171 


13 


0 


0 




7 


0 


0 


'r;ii75 


78 


12 


0 


^Vl17'^ 


122 


8 


0 


TH175 


7 


1 


0 


Tni76 


7 


0 


0 


1^1177 


36 


2li 


0 


•rH178 


57 

21 


9 


0 


■nti79 


1 


6 




k^} 


19 


0 




53 


7 


0 


T'Il82 


13 


6 


0 


Tiri85 




0 


0 


THl8i^ 


1 


0 


0 


Trii85 


2 


0 


0 


111186 


1 

J.. 


c 


0 


Trii87 




3h 




TnL88 




10 


0 


111189 


7 


1 


0 




12 




0 


TH,191 


10 




0 


•T:il9i".; 


7 


0 


0 


DLL 


2017 (as 


restart) 
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TABTil XIVlD 

Data on Use of Short Forais of Rules Summed over 
Students and Exercises (Fall, 1975) 



Rule 


No. of 
times used 


No. as 
short forms 


Error in 
short forms 


GA 


27U5 


205U 


20 


AS 


619 


65 


0 


Z 


1352 


576 


3 


N 


1228 


663 


k 


AI 


1033 


k06 


2 


U 


251 


0 


0 


NS 


552 


57 


1 


AD 


57 


0 


0 


TR 


163 


7 


0 


CN 


124 


12 


0 


DG 


138 


lb 


0 


NL 


73i+ 


0 


0 


NG 


561 


0 


0 


UI 


713 


5OI1 


0 


CU 


1030 


939 


10 


DI 


762 


311 


2 


DU 


612 


27U 


1 


II 


591 


289 


1 


RA 


679 


210 


0 


EM 


650 


251 


1 


tiA 


15 


1 


0 


SA 


lh3 


11 


0 


CS 


15U 


37 


0 


lA 


230 


40 


0 


THl 


528 


19U 


2 


TH2 


575 ■ 


269 


5 


TH5 


192 


63 


0 


THl^ 


177 


39 


0 


TH5 


198 


53 


0 


th6 


9k 


37 


2 


THT 


5a 


23 


0 


TH8 


5 


0 


0 


TH9 


1 


0 


0 


THll 


1 


0 


0 


THl 3 


5 


1 


0 


TH17 


1 


0 


0 


TH20 


1 


1 


' 0 


TH21 


2 


0 


0 


TH22 


2 


0 


0 
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ERIC 



(Table XT/b, cent.) 



No* of %o. as Error in 

times used - rt forms short forms 



LiiOU 


1 


u 


\j 


riiol 


10 7 


X 


r\ 


TXT/ O 


0 


PI 




i tio ^ 


J. 


U 


C\ 
yj 


i nO^ 


X 




0 


rrrtj l. 

1 nC^ 




n 
u 


D 




5 


U 


u 


i'^I00 


■ 3 


J 


u 




J 


0 


•J' 




yy 


*J 




.T , t 






'J 






J 


u 






ill 


/^ 
U 






T 
X 


U 










1 i-i ioU 






(L 








r\ 
U 




29i 




u 


Tlilc ] 




tr "1 


u 






0) 


u 


i t 9 


ib 




u 


I'M 7 ) 


1* ' 


'J 




•"T : i' ^ '1 "1 

4 ■ i ' 1 


■) 


,-\ 
J 


■J 


T'Iii72 


), 




u 


I HIT,") 




1 , 


U 


I H i /"i^ 




t 


.A 


H i i 




0 


0 


i M !. "'^^ 




■J 


/A 

u 


r!i.;77 


1 1 il 




U 


Till 'O 


29 




U 


THl i'9 


J. 


0 


0 


THIS) 






0 


nilO- 




9 


0 


TiU8^ 


') 




0 




i 


0 


■ 0 


I-H184 


2 


0 


0 


THl8^ 


1 


u 


0 


TH186 






0 


TH187 


89 


18 


0 


:hi88 


58 


10 


0 


THI89 


19 




0 




18 


6 


c» 




ic 


8 


0 




5 


0 


,) 


d;.l 


2601 


(as restart) 





85 



Most theorems, If used frequently, do not have high error rates. 
The ones the students did use frequently were those useful in proving 
that their interpretations of arguments (Lesson 428) were correct: TH6I 
and TH62« Of these, only WSl jhows a noticeable error rate in the 
spring class— mainly syntax errors. The students' attempts to use the 
short forms of the theorems met with cok: iderable success in terms of 
the number of errors. 

Of the quantifier rules, students demonstrated the most difficulty 
in learning existential specification (ES), as is clearly seen from the 
data in Table XIII. The errors were mostly application errors— selecting 
an ambiguous name that is either not well formed or is already introduced 
in the proof. The errors in UG, US, and EG were also application errors- 
selecting a term that cannot be used as a variable of generalization or 
attempting to specify a term such that the term contains a free occur- 
rence of a variable that will be capt\ired by a quantifier using that 
variable. Application errors in using the first quantifier negation 
rvle were also high. This is probably due to some confusion in deter- 
mining the scope of a, quantifier and of & negation symbol. Profiles of 
individual error histories were also constructed, but are not included 
for reasons of space* 

Student-defined Rules 

One of the connands the student can use is INIT, a reqiiest to select 
a different problem from the cuirriculxmi or to make up his own problems. 
The problem the student invents can be a derivation (DERIVE coomand) to 
test out his own notions about what constitutes a problem, or a proof 
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(PROVE cormuand) to prove a lemma that can help in completing subsequent 
curriculiOT problems. Using PROVE, the student constructs a proof of 
some well- formed formula and then provides a name, a label with which 
he later refers to the formula. 

Three significant statements can be made from these data. 

(1) Students did take advantage of INIT mode (an average of approxi- 
mately kl times in the spring and 22 in the fall), and they did use this 
mode to extend their command language for constructing proofs. Twenty- 
one out of 38 fall students and 25 out of hi spring students ccrapleted 
lemmas. Of these, Ik fall and 15 spring studentls received grades of A. 
J'^very student used his lemmas at least once in subsequent derivation 
problems. An average number of 5.^ (5*9 in the fall) lemmas were proved; 
these were used an average number of 17.5 times (28.7 for fall students). 

(2) The students, without exception, never made an error in using 
a lemma (command) they proved, 

(5) The names of the lemmas, except for a few cases (students 285O, 
P856, 2857, 2861, and 2867— all spring students), are all nonsense names. 
Vs.find anything from mmm to PREHISTORICMECHANIGALBEAST, from ALLDONE- 
WITHFINDINGAXIOMS to WOWIFITHSHEDIT, and from swear words to names of 
politicians. The semantic significance of this use of nonsense names 
certainly needs further investigation. The same use of nonsense words 
was prevalent in the finding-axioms exercises in naming axioms and 
theorems . 

Not all the students, of course, used IKIT mode to make up problems. 
But those that did used INTT to 
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(1) try out the sugfi^estetl proofs i'vcra Iiesson klQ, e.g., 

B+C=D -i B=D-C 
B=D-C -» BfC=D 
B+C=0 -» B=-C ; 

(2) make up lemmas to help in completing interpretation protlems, 

e.g., 

(E X) (X=X 8e X<5) 

NOT (A X)(X=X -» NOT X=X) ; 

(3) prove new formulas from Boolean algebra. 

Experimentation with the proof -checking program was, however, mini- 
mal in contrast to the program's actual design goals. l?he course is non- 
trivial; it takes time to complete all 53 lessons and 7 finding-axioms 
exercises. Because course grades are assigned according to the number 
of lessons completed, the students were anxious to complete the assigned 
curriculum and hesitated to spend time on extracurricular problems. 

Two Student Profiles 

In this section we attempt to identify characteristics of the stu- 
dents' work that reflect salient individual differences. We do this by* 
analyzing the profiles of two students. 

Both students were in the spring class and both received grades of 
A. That is where the similarity ends. One was the f .rat student to com- 
plete the course ♦ the other almost ijhe last. One always completed les- 
sons faster than the average time, the other always slower. Or^e spent 
time making up his own problems (and helping debug the curriculum), the 
other stayed vltnin the framework of stored currlculiim. One worked 

afternoons, the ether late at night. 
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More r>yr.te:nri^;jr;a>.Ty, we exai:..I *:ho lin.a records of the two st in- 
dents for the following features. (We have also commented parenthetically 
how information about each feature might be used to improve the coiirseO 

1. Error rate in usinf:'; rules. (This Iriformation is needed for 
modifying the choice of problems.) 

2* tYequency of syntax or application errors • (Application errors 
can be further analyzed in terms of knowledge of kinds of well-formed 
formulas^ pa^ern recognition, or understanding of the restrictions of 
the rule • ) 

3. Number of steps in completed solutions relative to the average* 
'If student fails to see and use rules or methods for shortening his 
derivations, some additional discussion might be appropriate, ) 

4* Use of riLLes of inference. (Does he become confused if pre- 
sented with a problem not requiring recently proved or learned rules?) 

'j. Time to completfj lessons and deviation from the average • (Does 
he need more or less praciice, or should time be ignored conpletely in 
favor of error rates?) 

6. Request for hints. (Is the student se ..ing a quick path, through 
the curriculum and looking for ansvers?) 

7. Use of INIT mode. (Does this usage affect the length of time 
spent on a lesson?) 

8. Proof of lemmas, (Does student use them to help solve the more 
difficult exercises?) 

9. Lata on choice C5f ^lerive, truth analysis, or interpretation 
mode. (Does student need more such choice exercises to develop his 
intuition about logical validity?) 
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In terms of the ihovt- i'eat^u.'es, ri\e • -v^Uer. of the- two students 
ve have selected are distinct, except for the number of hints requested. 
The fact that the student wbD was performing so well requested so many 
hints confirms our observation that he was interested in 'racing through* 
the curriculum for credit because his strong mathematics background in- 
cluded prior study of the material. We label the students 'A' and 'B*j 
Table XV highlights the differences in their performance. 

Insert Table XV about here 

Student A encountered several curriculum errors because he was 
usually the first student to reach a lesson. This probably accounts 
for the high n^jmber of sessions and the need to use the LESSON command 
to skip over problems that were incorrectly stated. He still spent 
less than the average amount of time, doing lessons in less time than 
the average, and making few errors in rule usage. Rule ES (existential 
specification) gave him trouble, but still less than it gave other stu- 
dents. His intuition on doing DI problems was comparatively poorer than 
average. He did make good use of INIT mode, especially for interpreta- 
tion and Boolean algebra problems. He vas, in fact, the only one to use ' 
INIT to prove leimnas in the Boolean algebra. He was not necessarily in- 
terested in finding minimum proofs, doing about the average number of steps. 
And, finally, he had to learn only once that he would have to follow the 
constraints set in the curriculum for each problem. This computer-based 
course suited Student A because he could finish early and concentrate on 
other courses. 
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lABIE XV 
Profiles of Two St\«ients 



Featxire 


student A 


Student B 


Grode 


A 


Tneantjletei finished With A 
grade the next quarter 


Progress 


Finished first 


Finished with A grade the next 
quarter 


Number of sessions 


96 (above average) 


286 (excessive) 


Time spent 


UI.5 hours (below average) 


111.8 (excessive) 


Deviation from 
average 


Always faster than average 


Usually slower 


Work hours 


U:00 p.m. to 5:00 p.m. preferred 


Nights, 8:00 p.m. or 9:00 p.m., 
11:00 p.m. to midnight 


Rule usage 

'j^ errors (Average 
for f errors) 
AR (17) 
CP (9) 
IP (17) 
^'O (6) 
US (6) 
EG (c)) 
ES (25) 


Generally better than average 

k 
0 
0 
7 
5 
7 

Ik 


Except for AR, always worse than 
average 

12 

19 

k9 

13 
15 

1^2 


C3iolce problems 
correct on lat 
or 2nd try) 

DC 
Die 


(Data 3ott due to eystem crash) 

57*< 
10056 






70 bints requested; 
62$ not available 


63 hints requested; 
61^ not available 
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(TJftble XV, cont.) 



Featiare ! 


Student A 


Sttdent B 


Short forms or 
axlcms and 
theorems 


Average amount for both axioms:, 
and theorems 


Only used for axicxns 


iniT tuode 
times types 
(21 averai^e) 

No. of DERIVES 

Ko« of PROVES 

No* PROVES 
completed 
(5 avera*^e) 

No. times Tiemmas 
usod 

No» times short 
form used 


70 

U 

20 
56 

1 1 


50 
0 
0 

\j 


No. of stepv^ to 
solution 


Average or below average 


Fluct\iated a lot, often finding 
minimum or average proofs but 
as often doing the maximtm 


L^'OSON conimard 
times requested 
(11 average) 




0 


REVIEW 

(Uj average) 


28 




REDO 

total no# times 
hid to redo 
problem 


1 


8 
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StAident B was a slow reader* ;f taKJ:;?* n low/ time to do even the first 
two lessons • He experienced a lot of trouble doing the lessons on inter- 
pretation and quantifier rules and always spent more time than the other 
students on each lesson^ He spent over 100 ho^irs on the course, more 
Lhan double the avera^^e, working mainly at night (when a teaching assis- 
tant was, unfortunately, not available to help him). All the rules gave 
him some trouble; IP and ES errors were application errors. We include 
his cumulative error curve for IP as an example (Figure l8); its approxi- 
mate linearity indicates little improvement from additional use of the 
rule. He never ventured to complete his own lemmas and never deviated 

Insert Figure l8 about here 

rrom the assigned, linear sequence of problems (despite the fact that 
redoing some problems might have given him needed reviews )• 

Alt' ough the course was often a frustraVing experience for Student 
B, he would almost certainly have received a low grade in a nonindividu- 
all7,ed, non- computer-based course , He had the opportunity to interact 
with and complete every problem. It took some prodding to convince him 
to continue; but he did, receiving a deserved grade of A. 
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ome Tactual i:ir..>t'ir:at. lO;.- 

1. What grade did you receive the coL-rseV 

vmat ,^^,rade aid you expeo:, r.o receive w^r-n you started the 

course? 

% Hjv did you t'ini. out al:oat r^he courrjeV 

■'u Did you find having the TICT ruL^ uf^efui; If not, why not? 

f . ^ '^^ere you jlvayo awar^'^ thd^ HIuT wa^ legal command In 
construe tiruT proof 3 , 

6^ Do you think the problems ncttea more hints-:? 

7. Did you attend the T'uesaay GeminarH? If not, why not? 

Problem types. 

1 I'er ivaticns and proof::; 

2 l^ru^n ^nalyci^ 

3 '"'Dunterexamples u^in^.^ truth analysis 
U Derive or Rive a ooa>'.ter?*xample 

') ;ouuterexampio by interpretation In the 

domain of r-vtional rpjmbers 
r- liprive ^o Ghov ^T^^jment valid or give an 

ln»>>rpretcit lo^' o r^hov invalid 
'/ 'Interpretation to shov premises consistent 
S .-•erivi: to shov premir-^^': inconsistent or 

give Interpretation to show con£?i?^tent 
9 jlntE;rpretatiDn to ^hov axicans lr;depe:ndent 

10 Derive to show axi^jm:^ dependent or give 

interpretation to show indepi^-ndent 

11 'Pranalate English sentences into first-order 

Iofjic'--dorive to mow answ^'-rr^ logically 
equivalent ("A'* people o^l>') 
Id rinding-axiom»s exerclseL^ 

V/hlch problem tj^pe was the eaGle^t? 

2. Which problem type the hardest? 

5, V/hich problem types did you like bei-t; 
4. V/hich problem type>^ aia you like ^eaotv 

Which specific problems^ i: an:y , uia you think were too hard? 
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III. Please read ea^^M rt,atet:ie::l :m'A oi. : nv^i^'^^er on the scale 

that best dei^'-'^Ml yo;ir v . 

bcale 

Moderately ar;ree 

U Uncertain 

S Slightly dlsap;ree 

Moderately disagree 
7 Strongly dl^^agree 

1, T th-ir.k I learnea fivjni the computer lessons 12 5^367 
well as I would havr- Ir^srned the name 

le^ sons i n tlx- cla"^ vroom . 

I liV.e worRlng at my own pace at the 1 2 5 U 5 6 7 

term inal. 

5* 1 prefer homework to working --^n problems 1 2 5 i| 5 6 7 

at trie tei^mlnal- 

V. 1 vov/id prefer comprLing with my fellow 1 2 5 4 5 6 7 

students in the classroom rather than 
wor-kinr at computer .'i-ssons, 

T find it frustrating rjt Knowing?; where 1 2 k 6 1 

my fellow classmater= arc In the lessons, 

//-^rkin^: with computer lest^ons 1:: like 12 5^5^7 

havin^^ my own tutor . 

Five hour=^ a week iG sufficient time to 125^5^7 
keep up with the coun'^e,. 

8. I found the computer lessons too easy. 1 2 5 5 6 7 

9* 1 think working with computr^r lessons is 125^567 
an exciting way to learn. 

IC* I found working at the terminal 12 5^+567 

f rustrat ing« 

11. I would like to participate in another 12 3 4 5^7 
computer-based course. 

12. 1 found the computer l<:S3ons too hard. 1 2 5 U 5 6 7 

15. The ::jnputer provi the student with 12 5^+ h 7 

more feedback than classroom instruction. 



lU* The terminal^: were ava i Lixblo to me when 1 i C ^ k ') 
wanted to work* 

1^. There vras sufficient outside help when 1 2 5 7 

I needed it# 

l6. Use the back of this sheet to make any 
comments yc\i wish concernln^^ the course. 
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38 
59 

1*0 

1*1 

1*2 
1*1+ 

1*6 
1*9 

13 
11* 

15 
i6 
17 
18 
18 
19 
19 
20 
21 
21 
22 
22 
23 
25 
21* 

25 
26 



haiidom 
choice 

1 
2 
1 
2 
0 

0 
0 

1 

2 
1 
2 
1 
2 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 

0 

c 

0 
0 
0 

1 

2 

1 

2 
0 

1 

2 
1 
2 
1 
2 
0 
0 
0 



Timber of steps 



Min 

9 
12 

9 

10 

7 

2 
2 

5 

11 

7 

11 
i* 
2 
6 
5 
5 
5 
5 

U 
U 
8 
9 
5 
5 
8 

3 
3 
3 

5 
5 
5 
U 

5 
7 

11 
U 
7 

7 

1* 
6 
21^ 

5 

6 



Max 

29 

11 

17 
17 

5 
7 

21* 
2^) 
20 
20 

6 

U 

6 

19 
6 
6 
6 
7 

7 
25 
^1* 

9 
8 
18 

9 

5 
10 
12 

6 

9 
10 
10 
11* 
17 
20 
18 

13 

16 
11* 
29 
6 

12 
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ERIC 



Les- 
son 


Prob- 
lem 


Handom 
choice 




r> 1 * . [. t f 


* _ 








W ^ ft- V W »~» Ik? 


Mill 


Max 










r 


0 


1^ 


14 


4 5^ 


i. > 


n 


•t 


Q 


IC 


0 


* 
5 


Q 

0 






C\ 








11 


0 








i i 


n 

V 








12 


0 


5 


19 






0 


4 






13 


0 


5 


26 




ok 




D 






lU 


0 


2 


15 




2^> 


0 


5 


1*2 




17 


c 




1 

4 




27 


0 




55 






c. 




6 




2o 


0 


0 








c 


7 


19 




50 


c 


17 








0 


8 


2S 




33 


0 


1^ 


37 






0 


1 

k 


19 




34 


0 


14 


55 














37 


0 


1 


26 






u 


c 
7 




38 


0 


3 


20 








7 


w 




kl 


0 


6 


15 




i 


c 




15 




k2 


0 


5 


Ik 




.'i 


0 


X 






hi 


0 


5 


lU 




§'-\ 
lO 




1 


•* 

5 




kk 


0 


lU 


58 




1''*- 


c 


D 


Ix 






0 


10 


28 




. I 1 


0 


2 


7 




46 


0 


8 


59 






l" 


7 


lo 




1*7 


0 


26 


95 






0 


1 


15 












*J 


7 


2U 






0 


2 


5 






(1 


ill 


1"; 




6 


0 


3 


12 






0 


7 


1^^ 




7 


0 


U 


2V 






0 


9 


26 




8 


0 


21^ 


57 






0 


6 


8 




9 


0 


9 


50 






c 


? 


10 




10 


0 


11 


30 






c 


11 


18 




12 


0 


6 


63 








k 






I'i 


0 


7 


57 










12 




\h 


0 


11 


lf6 






c 


5 








6 




9 


0 


5 


9 




11 


0 


5 




16 


0 


9 


55 




12 


0 


3 


8 




11 


0 


8 


29 




13 


0 


3 


19 




12 


0 


5 


20 




16 


0 


7 


52 




15 


0 


5 


15 




17 


0 


7 


21 




14 


0 


4 


23 
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Logical Vule>z or ;nii"ere!ice and Frooi' Procedures 



(In the followiru?, i, ,j, and k are variables for line numbers.) 
l) Affirm the antecedent 

i) P ->v^, 

j) P 

K) C. 



2) .'oinmute conjuncts 

i ) P Q 
i'"Cl :0 Q &• P 



5^ ^.''ommute disjuncts 

i) P OR Q 
i • : -1 j ) Q OR P 



'■ "/ ""jiTLTiute equals 

i) B = C 
i-.:t;i ,1) c = B 



.'.jnanute equivalence 
Xl Q IFF R 

i:.ra J) R iFP' Q, 



6) Conditional proof procedure 

WP i) R line 1 must be the 

j ) Q, last working premise 

l.jCP k) R -^Q introduced 



7) Contrapositlve 

i) R -^Q 
ICON J) (NOT Q) (NOT R) 



105 



S) Deny the eoncequei. 



1 ) p -> i ^ 1 ■ . NO'] 



9) Deny a disjunct 

i) H OR Q i) R OR Q 

j ) NOT R j ) NOT Q 

l.JDD k) Q k) R 

10) Ic-ri'-iitJ. ■..■'!. of implication 

i) 0 -> K 1) (NOT Q: 

iPFA J) fNOT Q) OH R j) Q -*R 



11) '\- Moi-f-an's Law 

O HOT (R OR Q) i) NOT (R ft. Q) 
IDr-T 1 ^lOT H ft NOT Q j ) NOT R OR NOT Q 



1^ I Dou: . ! r r -?;?:at i on 

■; ) i) NOT (NOT R) 

nr:: :) not (NOT r) j) r 



13) !'''c!^.rjV'iJt'3 ner;ation over implication 

: ■; ^v, -> t,') i) Q, ft NOT R 

,0 Q ft NOT R J) NOT (Q R) 



Ik) Disjunctive syllogism 

i) Q OR K 

j) Q ->P 

k) R -^W 

i. j.kDS 

m) P OR W 



15) Existential generalization 
i) F(*) 

iEG 

»:X j) (E X) F(X) 
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16) Existential ■ \'i --ui ioi. 
i) (EX) V(X,Y) 

lES 



17) Form a cor.,ii:nctlon 
i) R 

i.,1FC k) R Q 



ly) I'orm a disjunction 
i) R 

IFD 

: :r, J) R OR Q 



19/ Hypothetical syllogism 

i) P -^R 



20) Indirect proof procedure 

line i must be 
the last working premise 
introduced 



W i) R 

o) 

■ k) NOT Q 
1..1,kIP m) NOT R 



21) Law of the biconditional 

i) R IFF Q i) (R -»ft) & (Q ^R) 

iLB J) (R -^Q) & (Q -*R) .j) R IFF Q 



22) Left conjunct 

i) R & P 
ILC J) R 



23) Logical truth 
LT 

::X i) X = X 
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2k ) Quant 3. f i e r- t ;e fa x. \ ■:. ; 



i) (A x:> i>;x; ^. ; ^ ri (:•: 0 i,'OT : 

iQNA j) NOT (F. X) NOT I'm'X) .? ) (A X) r3(>:) 



25) Quantifier negatiori rule B 

i) (A X) r^OT S(X) i) NOT (K X) i^iX) 

iQNB j) NOT (E X) S(X) j) (A X) NOT 3(X) 



janti f ifr nei'-.ation rule C 

i ) ! [•: y) 3{X) i) NOT (A X) NOT 

iQNC jj MOT {Z X) NOT S(X) j) (E X) S(X) 



27) Ouantifier negation rule D 

i) :e X) NOT S(X) i) NOT (A X) S(X) 

iQ?.T .1) (A X) S(X) j) (EX) NOT S(X) 



?8} :-i..:ht conjunct 

1 ^ [\ & p 

i ) P+c =^ B+C 
■i.jKi;.. ij iHC - 3 

30) Universal generalization 

i) FfX) 

iUG 

:X j) (A X) F(X) 

51) Universal specification 

i) (A X) F(X) 

iUS 

X:Y J) F(Y) 

52) Working premise 
WP i) R 
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1) COPY a line 

i) R (useful for checkirip; on what the computer 
ICOPY j) H thirJ^s yo^; typed) 

2) Delete lines 

IDLL deletes all lines beginning with line 1 
5) vZ.U logs you off the computer 

'4) Hypothesis 

HYP creates a working premise for you 

If the statement 

of the problem is HYP gives 

E -->Q R 
TIOT R R 

R NOT R 

5) Initiative (is available at most points at 

which you are to type a response) 

TNTT ^ lets you ask for your own problems j 

program will always return to the problem 
you interrupted 

you can request 

DERIVE (request a derive problem — you may use 

for PREMISE^ as the initial commands) 

FA (request finding-axioms exercise) 

LESSON (request a different lesson and problem) 

NEWS (request the news of the day, i#e#, computer 

schedule,, program changes, class meetings; 
also available when you are constructing 
a derivation) 

PROVE (when a PROVE problem is completed, you 

can name the expression as a theorem) 
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REVIEW the computer will type ea-}i ccr^jnat.d 3.:vi proof line, 
any flap;fred varial^les, and the premise lines on 
which the t lagging depends 

7) Requests for problem types when you are given T^ht.- choice 

CEX counterexscnple problem 

counterexample by assignment of truth values 

!?KK derivation problem 

INT' interpretation problem 

8) H'^'wTART (available only when doing 

interpretation problems) 

will let you change your Interpretation 
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